Oct 30 21:52:19.239417 ip-10-0-131-202 systemd[1]: Starting Kubernetes Kubelet...
Oct 30 21:52:19.658265 ip-10-0-131-202 kubenswrapper[2578]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 30 21:52:19.658265 ip-10-0-131-202 kubenswrapper[2578]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Oct 30 21:52:19.658265 ip-10-0-131-202 kubenswrapper[2578]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 30 21:52:19.658265 ip-10-0-131-202 kubenswrapper[2578]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 30 21:52:19.664162 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.664041    2578 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 21:52:19.664162 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.664143    2578 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 21:52:19.664162 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.664149    2578 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 21:52:19.664162 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.664153    2578 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 21:52:19.664162 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.664160    2578 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 21:52:19.664162 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.664165    2578 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 21:52:19.664162 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.664168    2578 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 21:52:19.664162 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.664172    2578 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 21:52:19.664437 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.664175    2578 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 21:52:19.664437 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.664178    2578 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 21:52:19.664437 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.664181    2578 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 21:52:19.664437 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.664184    2578 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 21:52:19.664437 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.664186    2578 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 21:52:19.664437 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.664189    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 21:52:19.664437 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.664191    2578 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 21:52:19.664437 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.664194    2578 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 21:52:19.664437 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.664205    2578 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 21:52:19.664437 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.664210    2578 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 21:52:19.664437 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.664214    2578 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 21:52:19.664437 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.664217    2578 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 21:52:19.664437 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.664220    2578 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 21:52:19.664437 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.664223    2578 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 21:52:19.664437 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.664225    2578 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 21:52:19.664437 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.664228    2578 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 21:52:19.664437 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.664231    2578 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 21:52:19.664437 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.664233    2578 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 21:52:19.664437 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.664236    2578 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 21:52:19.665017 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.664238    2578 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 21:52:19.665017 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.664241    2578 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 21:52:19.665017 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.664243    2578 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 21:52:19.665017 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.664247    2578 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 21:52:19.665017 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.664250    2578 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 21:52:19.665017 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.664252    2578 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 21:52:19.665017 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.664255    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 21:52:19.665017 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.664257    2578 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 21:52:19.665017 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.664260    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 21:52:19.665017 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.664262    2578 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 21:52:19.665017 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.664265    2578 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 21:52:19.665017 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.664267    2578 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 21:52:19.665017 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.664270    2578 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 21:52:19.665017 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.664273    2578 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 21:52:19.665017 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.664277    2578 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 21:52:19.665017 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.664282    2578 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 21:52:19.665017 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.664285    2578 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 21:52:19.665017 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.664287    2578 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 21:52:19.665017 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.664290    2578 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 21:52:19.665017 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.664293    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 21:52:19.665668 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.664295    2578 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 21:52:19.665668 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.664298    2578 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 21:52:19.665668 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.664301    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 21:52:19.665668 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.664304    2578 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 21:52:19.665668 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.664306    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 21:52:19.665668 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.664308    2578 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 21:52:19.665668 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.664311    2578 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 21:52:19.665668 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.664314    2578 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 21:52:19.665668 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.664316    2578 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 21:52:19.665668 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.664319    2578 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 21:52:19.665668 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.664321    2578 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 21:52:19.665668 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.664324    2578 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 21:52:19.665668 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.664326    2578 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 21:52:19.665668 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.664329    2578 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 21:52:19.665668 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.664331    2578 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 21:52:19.665668 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.664333    2578 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 21:52:19.665668 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.664336    2578 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 21:52:19.665668 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.664339    2578 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 21:52:19.665668 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.664341    2578 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 21:52:19.666131 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.664344    2578 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 21:52:19.666131 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.664346    2578 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 21:52:19.666131 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.664349    2578 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 21:52:19.666131 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.664352    2578 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 21:52:19.666131 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.664355    2578 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 21:52:19.666131 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.664358    2578 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 21:52:19.666131 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.664361    2578 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 21:52:19.666131 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.664364    2578 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 21:52:19.666131 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.664367    2578 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 21:52:19.666131 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.664370    2578 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 21:52:19.666131 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.664372    2578 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 21:52:19.666131 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.664375    2578 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 21:52:19.666131 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.664377    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 21:52:19.666131 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.664380    2578 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 21:52:19.666131 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.664382    2578 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 21:52:19.666131 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.664385    2578 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 21:52:19.666131 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.664388    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 21:52:19.666131 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.664390    2578 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 21:52:19.666131 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.664393    2578 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 21:52:19.666588 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.664395    2578 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 21:52:19.666588 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.664399    2578 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 21:52:19.666588 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.664401    2578 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 21:52:19.666588 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.664404    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 21:52:19.666588 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.664407    2578 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 21:52:19.666588 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.664409    2578 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 21:52:19.666588 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.664412    2578 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 21:52:19.666588 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.664414    2578 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 21:52:19.666588 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.664417    2578 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 21:52:19.666588 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.664419    2578 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 21:52:19.666588 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.664422    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 21:52:19.666588 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.664424    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 21:52:19.666588 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.664427    2578 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 21:52:19.666588 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.664429    2578 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 21:52:19.666588 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.664432    2578 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 21:52:19.666588 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.664434    2578 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 21:52:19.666588 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.664437    2578 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 21:52:19.666588 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.664440    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 21:52:19.666588 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.664442    2578 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 21:52:19.667067 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.664446    2578 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 21:52:19.667067 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.664449    2578 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 21:52:19.668008 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.667985    2578 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 21:52:19.668008 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.668006    2578 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 21:52:19.668008 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.668011    2578 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 21:52:19.668245 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.668015    2578 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 21:52:19.668245 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.668019    2578 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 21:52:19.668245 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.668023    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 21:52:19.668245 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.668027    2578 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 21:52:19.668245 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.668030    2578 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 21:52:19.668245 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.668034    2578 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 21:52:19.668245 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.668038    2578 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 21:52:19.668245 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.668042    2578 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 21:52:19.668245 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.668046    2578 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 21:52:19.668245 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.668050    2578 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 21:52:19.668245 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.668054    2578 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 21:52:19.668245 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.668057    2578 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 21:52:19.668245 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.668063    2578 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 21:52:19.668245 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.668067    2578 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 21:52:19.668245 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.668071    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 21:52:19.668245 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.668075    2578 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 21:52:19.668245 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.668079    2578 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 21:52:19.668245 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.668084    2578 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 21:52:19.668245 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.668088    2578 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 21:52:19.668245 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.668092    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 21:52:19.669081 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.668096    2578 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 21:52:19.669081 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.668100    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 21:52:19.669081 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.668105    2578 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 21:52:19.669081 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.668108    2578 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 21:52:19.669081 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.668113    2578 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 21:52:19.669081 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.668140    2578 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 21:52:19.669081 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.668146    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 21:52:19.669081 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.668150    2578 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 21:52:19.669081 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.668155    2578 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 21:52:19.669081 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.668159    2578 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 21:52:19.669081 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.668164    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 21:52:19.669081 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.668168    2578 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 21:52:19.669081 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.668172    2578 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 21:52:19.669081 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.668177    2578 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 21:52:19.669081 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.668181    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 21:52:19.669081 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.668186    2578 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 21:52:19.669081 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.668190    2578 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 21:52:19.669081 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.668194    2578 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 21:52:19.669081 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.668198    2578 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 21:52:19.669929 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.668202    2578 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 21:52:19.669929 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.668207    2578 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 21:52:19.669929 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.668211    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 21:52:19.669929 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.668215    2578 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 21:52:19.669929 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.668219    2578 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 21:52:19.669929 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.668224    2578 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 21:52:19.669929 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.668228    2578 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 21:52:19.669929 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.668232    2578 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 21:52:19.669929 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.668237    2578 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 21:52:19.669929 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.668241    2578 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 21:52:19.669929 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.668245    2578 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 21:52:19.669929 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.668249    2578 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 21:52:19.669929 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.668253    2578 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 21:52:19.669929 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.668257    2578 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 21:52:19.669929 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.668263    2578 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 21:52:19.669929 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.668267    2578 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 21:52:19.669929 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.668271    2578 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 21:52:19.669929 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.668276    2578 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 21:52:19.669929 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.668281    2578 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 21:52:19.669929 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.668285    2578 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 21:52:19.670506 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.668292    2578 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 21:52:19.670506 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.668298    2578 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 21:52:19.670506 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.668303    2578 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 21:52:19.670506 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.668307    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 21:52:19.670506 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.668311    2578 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 21:52:19.670506 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.668316    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 21:52:19.670506 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.668320    2578 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 21:52:19.670506 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.668326    2578 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 21:52:19.670506 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.668330    2578 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 21:52:19.670506 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.668334    2578 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 21:52:19.670506 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.668338    2578 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 21:52:19.670506 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.668342    2578 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 21:52:19.670506 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.668346    2578 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 21:52:19.670506 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.668350    2578 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 21:52:19.670506 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.668354    2578 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 21:52:19.670506 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.668358    2578 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 21:52:19.670506 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.668363    2578 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 21:52:19.670506 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.668367    2578 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 21:52:19.670506 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.668371    2578 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 21:52:19.670506 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.668375    2578 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 21:52:19.671007 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.668380    2578 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 21:52:19.671007 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.668385    2578 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 21:52:19.671007 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.668390    2578 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 21:52:19.671007 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.668394    2578 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 21:52:19.671007 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.668398    2578 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 21:52:19.671007 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.668403    2578 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 21:52:19.671007 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.668407    2578 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 21:52:19.671007 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.668412    2578 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 21:52:19.671007 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.668416    2578 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 21:52:19.671007 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.668420    2578 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 21:52:19.671007 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.668424    2578 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 21:52:19.671007 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.668429    2578 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 21:52:19.671007 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.668433    2578 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 21:52:19.671007 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.668437    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 21:52:19.671007 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.668442    2578 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 21:52:19.671007 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.668446    2578 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 21:52:19.671007 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.668450    2578 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 21:52:19.671007 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.668457    2578 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 21:52:19.671007 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.668464    2578 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 21:52:19.671498 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.668469    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 21:52:19.671498 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.668474    2578 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 21:52:19.671498 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.668478    2578 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 21:52:19.671498 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.668482    2578 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 21:52:19.671498 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.668486    2578 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 21:52:19.671498 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.669456    2578 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 21:52:19.671498 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.669468    2578 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 21:52:19.671498 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.669475    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 21:52:19.671498 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.669480    2578 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 21:52:19.671498 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.669485    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 21:52:19.671498 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.669489    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 21:52:19.671498 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.669494    2578 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 21:52:19.671498 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.669498    2578 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 21:52:19.671498 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.669502    2578 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 21:52:19.671498 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.669507    2578 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 21:52:19.671498 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.669512    2578 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 21:52:19.671498 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.669518    2578 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 21:52:19.671498 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.669522    2578 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 21:52:19.671498 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.669526    2578 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 21:52:19.671498 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.669530    2578 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 21:52:19.672153 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.669535    2578 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 21:52:19.672153 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.669539    2578 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 21:52:19.672153 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.669543    2578 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 21:52:19.672153 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.669546    2578 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 21:52:19.672153 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.669550    2578 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 21:52:19.672153 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.669555    2578 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 21:52:19.672153 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.669559    2578 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 21:52:19.672153 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.669563    2578 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 21:52:19.672153 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.669568    2578 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 21:52:19.672153 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.669572    2578 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 21:52:19.672153 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.669576    2578 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 21:52:19.672153 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.669580    2578 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 21:52:19.672153 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.669584    2578 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 21:52:19.672153 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.669588    2578 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 21:52:19.672153 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.669592    2578 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 21:52:19.672153 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.669597    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 21:52:19.672153 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.669603    2578 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 21:52:19.672153 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.669608    2578 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 21:52:19.672153 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.669612    2578 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 21:52:19.672977 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.669617    2578 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 21:52:19.672977 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.669621    2578 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 21:52:19.672977 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.669625    2578 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 21:52:19.672977 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.669629    2578 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 21:52:19.672977 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.669633    2578 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 21:52:19.672977 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.669637    2578 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 21:52:19.672977 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.669641    2578 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 21:52:19.672977 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.669646    2578 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 21:52:19.672977 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.669651    2578 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 21:52:19.672977 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.669656    2578 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 21:52:19.672977 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.669660    2578 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 21:52:19.672977 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.669664    2578 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 21:52:19.672977 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.669668    2578 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 21:52:19.672977 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.669672    2578 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 21:52:19.672977 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.669676    2578 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 21:52:19.672977 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.669680    2578 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 21:52:19.672977 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.669684    2578 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 21:52:19.672977 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.669688    2578 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 21:52:19.672977 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.669692    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 21:52:19.673871 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.669697    2578 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 21:52:19.673871 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.669701    2578 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 21:52:19.673871 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.669705    2578 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 21:52:19.673871 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.669709    2578 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 21:52:19.673871 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.669714    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 21:52:19.673871 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.669717    2578 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 21:52:19.673871 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.669721    2578 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 21:52:19.673871 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.669726    2578 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 21:52:19.673871 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.669730    2578 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 21:52:19.673871 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.669734    2578 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 21:52:19.673871 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.669738    2578 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 21:52:19.673871 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.669743    2578 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 21:52:19.673871 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.669747    2578 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 21:52:19.673871 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.669751    2578 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 21:52:19.673871 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.669756    2578 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 21:52:19.673871 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.669760    2578 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 21:52:19.673871 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.669764    2578 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 21:52:19.673871 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.669768    2578 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 21:52:19.673871 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.669773    2578 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 21:52:19.673871 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.669777    2578 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 21:52:19.674461 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.669782    2578 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 21:52:19.674461 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.669786    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 21:52:19.674461 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.669791    2578 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 21:52:19.674461 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.669795    2578 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 21:52:19.674461 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.669799    2578 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 21:52:19.674461 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.669803    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 21:52:19.674461 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.669807    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 21:52:19.674461 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.669814    2578 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 21:52:19.674461 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.669818    2578 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 21:52:19.674461 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.669822    2578 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 21:52:19.674461 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.669827    2578 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 21:52:19.674461 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.669832    2578 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 21:52:19.674461 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.669837    2578 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 21:52:19.674461 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.669841    2578 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 21:52:19.674461 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.669845    2578 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 21:52:19.674461 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.669849    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 21:52:19.674461 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.669854    2578 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 21:52:19.674461 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.669858    2578 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 21:52:19.674461 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.669862    2578 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 21:52:19.674461 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.669867    2578 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 21:52:19.675080 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.669871    2578 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 21:52:19.675080 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.669876    2578 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 21:52:19.675080 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.669880    2578 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 21:52:19.675080 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.669886    2578 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 21:52:19.675080 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.669895    2578 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 21:52:19.675080 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.669902    2578 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 21:52:19.675080 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.669907    2578 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 21:52:19.675080 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.669912    2578 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 21:52:19.675080 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.669918    2578 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 21:52:19.675080 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.669922    2578 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 21:52:19.675080 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.669927    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 21:52:19.675080 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.669931    2578 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 21:52:19.675080 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.669936    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 21:52:19.675080 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:19.671779    2578 flags.go:64] FLAG: --address="0.0.0.0"
Oct 30 21:52:19.675080 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:19.671797    2578 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Oct 30 21:52:19.675080 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:19.671811    2578 flags.go:64] FLAG: --anonymous-auth="true"
Oct 30 21:52:19.675080 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:19.671819    2578 flags.go:64] FLAG: --application-metrics-count-limit="100"
Oct 30 21:52:19.675080 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:19.671826    2578 flags.go:64] FLAG: --authentication-token-webhook="false"
Oct 30 21:52:19.675080 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:19.671831    2578 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Oct 30 21:52:19.675080 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:19.671840    2578 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Oct 30 21:52:19.675669 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:19.671848    2578 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Oct 30 21:52:19.675669 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:19.671853    2578 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Oct 30 21:52:19.675669 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:19.671858    2578 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Oct 30 21:52:19.675669 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:19.671864    2578 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Oct 30 21:52:19.675669 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:19.671869    2578 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Oct 30 21:52:19.675669 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:19.671874    2578 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Oct 30 21:52:19.675669 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:19.671879    2578 flags.go:64] FLAG: --cgroup-root=""
Oct 30 21:52:19.675669 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:19.671883    2578 flags.go:64] FLAG: --cgroups-per-qos="true"
Oct 30 21:52:19.675669 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:19.671888    2578 flags.go:64] FLAG: --client-ca-file=""
Oct 30 21:52:19.675669 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:19.671893    2578 flags.go:64] FLAG: --cloud-provider="external"
Oct 30 21:52:19.675669 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:19.671897    2578 flags.go:64] FLAG: --cluster-dns="[]"
Oct 30 21:52:19.675669 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:19.671903    2578 flags.go:64] FLAG: --cluster-domain=""
Oct 30 21:52:19.675669 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:19.671908    2578 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Oct 30 21:52:19.675669 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:19.671913    2578 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Oct 30 21:52:19.675669 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:19.671918    2578 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Oct 30 21:52:19.675669 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:19.671923    2578 flags.go:64] FLAG: --container-log-max-files="5"
Oct 30 21:52:19.675669 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:19.671930    2578 flags.go:64] FLAG: --container-log-max-size="10Mi"
Oct 30 21:52:19.675669 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:19.671936    2578 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Oct 30 21:52:19.675669 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:19.671941    2578 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Oct 30 21:52:19.675669 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:19.671946    2578 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Oct 30 21:52:19.675669 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:19.671951    2578 flags.go:64] FLAG: --contention-profiling="false"
Oct 30 21:52:19.675669 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:19.671955    2578 flags.go:64] FLAG: --cpu-cfs-quota="true"
Oct 30 21:52:19.675669 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:19.671960    2578 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Oct 30 21:52:19.675669 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:19.671965    2578 flags.go:64] FLAG: --cpu-manager-policy="none"
Oct 30 21:52:19.676266 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:19.671969    2578 flags.go:64] FLAG: --cpu-manager-policy-options=""
Oct 30 21:52:19.676266 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:19.671976    2578 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Oct 30 21:52:19.676266 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:19.671982    2578 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Oct 30 21:52:19.676266 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:19.671987    2578 flags.go:64] FLAG: --enable-debugging-handlers="true"
Oct 30 21:52:19.676266 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:19.671992    2578 flags.go:64] FLAG: --enable-load-reader="false"
Oct 30 21:52:19.676266 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:19.671997    2578 flags.go:64] FLAG: --enable-server="true"
Oct 30 21:52:19.676266 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:19.672002    2578 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Oct 30 21:52:19.676266 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:19.672010    2578 flags.go:64] FLAG: --event-burst="100"
Oct 30 21:52:19.676266 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:19.672015    2578 flags.go:64] FLAG: --event-qps="50"
Oct 30 21:52:19.676266 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:19.672020    2578 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Oct 30 21:52:19.676266 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:19.672025    2578 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Oct 30 21:52:19.676266 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:19.672029    2578 flags.go:64] FLAG: --eviction-hard=""
Oct 30 21:52:19.676266 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:19.672036    2578 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Oct 30 21:52:19.676266 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:19.672041    2578 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Oct 30 21:52:19.676266 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:19.672045    2578 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Oct 30 21:52:19.676266 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:19.672050    2578 flags.go:64] FLAG: --eviction-soft=""
Oct 30 21:52:19.676266 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:19.672055    2578 flags.go:64] FLAG: --eviction-soft-grace-period=""
Oct 30 21:52:19.676266 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:19.672060    2578 flags.go:64] FLAG: --exit-on-lock-contention="false"
Oct 30 21:52:19.676266 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:19.672064    2578 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Oct 30 21:52:19.676266 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:19.672069    2578 flags.go:64] FLAG: --experimental-mounter-path=""
Oct 30 21:52:19.676266 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:19.672074    2578 flags.go:64] FLAG: --fail-cgroupv1="true"
Oct 30 21:52:19.676266 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:19.672079    2578 flags.go:64] FLAG: --fail-swap-on="true"
Oct 30 21:52:19.676266 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:19.672084    2578 flags.go:64] FLAG: --feature-gates=""
Oct 30 21:52:19.676266 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:19.672090    2578 flags.go:64] FLAG: --file-check-frequency="20s"
Oct 30 21:52:19.676266 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:19.672095    2578 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Oct 30 21:52:19.676950 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:19.672101    2578 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Oct 30 21:52:19.676950 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:19.672106    2578 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Oct 30 21:52:19.676950 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:19.672111    2578 flags.go:64] FLAG: --healthz-port="10248"
Oct 30 21:52:19.676950 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:19.672134    2578 flags.go:64] FLAG: --help="false"
Oct 30 21:52:19.676950 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:19.672140    2578 flags.go:64] FLAG: --hostname-override="ip-10-0-131-202.us-west-1.compute.internal"
Oct 30 21:52:19.676950 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:19.672145    2578 flags.go:64] FLAG: --housekeeping-interval="10s"
Oct 30 21:52:19.676950 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:19.672150    2578 flags.go:64] FLAG: --http-check-frequency="20s"
Oct 30 21:52:19.676950 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:19.672155    2578 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Oct 30 21:52:19.676950 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:19.672160    2578 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Oct 30 21:52:19.676950 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:19.672165    2578 flags.go:64] FLAG: --image-gc-high-threshold="85"
Oct 30 21:52:19.676950 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:19.672170    2578 flags.go:64] FLAG: --image-gc-low-threshold="80"
Oct 30 21:52:19.676950 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:19.672174    2578 flags.go:64] FLAG: --image-service-endpoint=""
Oct 30 21:52:19.676950 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:19.672180    2578 flags.go:64] FLAG: --kernel-memcg-notification="false"
Oct 30 21:52:19.676950 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:19.672185    2578 flags.go:64] FLAG: --kube-api-burst="100"
Oct 30 21:52:19.676950 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:19.672190    2578 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Oct 30 21:52:19.676950 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:19.672195    2578 flags.go:64] FLAG: --kube-api-qps="50"
Oct 30 21:52:19.676950 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:19.672199    2578 flags.go:64] FLAG: --kube-reserved=""
Oct 30 21:52:19.676950 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:19.672204    2578 flags.go:64] FLAG: --kube-reserved-cgroup=""
Oct 30 21:52:19.676950 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:19.672208    2578 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Oct 30 21:52:19.676950 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:19.672213    2578 flags.go:64] FLAG: --kubelet-cgroups=""
Oct 30 21:52:19.676950 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:19.672217    2578 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Oct 30 21:52:19.676950 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:19.672222    2578 flags.go:64] FLAG: --lock-file=""
Oct 30 21:52:19.676950 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:19.672227    2578 flags.go:64] FLAG: --log-cadvisor-usage="false"
Oct 30 21:52:19.677584 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:19.672231    2578 flags.go:64] FLAG: --log-flush-frequency="5s"
Oct 30 21:52:19.677584 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:19.672236    2578 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Oct 30 21:52:19.677584 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:19.672246    2578 flags.go:64] FLAG: --log-json-split-stream="false"
Oct 30 21:52:19.677584 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:19.672250    2578 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Oct 30 21:52:19.677584 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:19.672255    2578 flags.go:64] FLAG: --log-text-split-stream="false"
Oct 30 21:52:19.677584 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:19.672259    2578 flags.go:64] FLAG: --logging-format="text"
Oct 30 21:52:19.677584 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:19.672264    2578 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Oct 30 21:52:19.677584 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:19.672270    2578 flags.go:64] FLAG: --make-iptables-util-chains="true"
Oct 30 21:52:19.677584 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:19.672275    2578 flags.go:64] FLAG: --manifest-url=""
Oct 30 21:52:19.677584 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:19.672279    2578 flags.go:64] FLAG: --manifest-url-header=""
Oct 30 21:52:19.677584 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:19.672287    2578 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Oct 30 21:52:19.677584 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:19.672292    2578 flags.go:64] FLAG: --max-open-files="1000000"
Oct 30 21:52:19.677584 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:19.672298    2578 flags.go:64] FLAG: --max-pods="110"
Oct 30 21:52:19.677584 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:19.672303    2578 flags.go:64] FLAG: --maximum-dead-containers="-1"
Oct 30 21:52:19.677584 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:19.672308    2578 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Oct 30 21:52:19.677584 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:19.672313    2578 flags.go:64] FLAG: --memory-manager-policy="None"
Oct 30 21:52:19.677584 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:19.672317    2578 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Oct 30 21:52:19.677584 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:19.672322    2578 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Oct 30 21:52:19.677584 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:19.672327    2578 flags.go:64] FLAG: --node-ip="0.0.0.0"
Oct 30 21:52:19.677584 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:19.672331    2578 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Oct 30 21:52:19.677584 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:19.672346    2578 flags.go:64] FLAG: --node-status-max-images="50"
Oct 30 21:52:19.677584 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:19.672351    2578 flags.go:64] FLAG: --node-status-update-frequency="10s"
Oct 30 21:52:19.677584 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:19.672357    2578 flags.go:64] FLAG: --oom-score-adj="-999"
Oct 30 21:52:19.677584 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:19.672363    2578 flags.go:64] FLAG: --pod-cidr=""
Oct 30 21:52:19.677584 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:19.672367    2578 flags.go:64] FLAG: --pod-manifest-path=""
Oct 30 21:52:19.678203 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:19.672372    2578 flags.go:64] FLAG: --pod-max-pids="-1"
Oct 30 21:52:19.678203 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:19.672377    2578 flags.go:64] FLAG: --pods-per-core="0"
Oct 30 21:52:19.678203 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:19.672381    2578 flags.go:64] FLAG: --port="10250"
Oct 30 21:52:19.678203 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:19.672386    2578 flags.go:64] FLAG: --protect-kernel-defaults="false"
Oct 30 21:52:19.678203 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:19.672391    2578 flags.go:64] FLAG: --provider-id="aws:///us-west-1a/i-02f3778ad02613422"
Oct 30 21:52:19.678203 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:19.672396    2578 flags.go:64] FLAG: --qos-reserved=""
Oct 30 21:52:19.678203 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:19.672401    2578 flags.go:64] FLAG: --read-only-port="10255"
Oct 30 21:52:19.678203 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:19.672406    2578 flags.go:64] FLAG: --register-node="true"
Oct 30 21:52:19.678203 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:19.672411    2578 flags.go:64] FLAG: --register-with-taints=""
Oct 30 21:52:19.678203 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:19.672417    2578 flags.go:64] FLAG: --registry-burst="10"
Oct 30 21:52:19.678203 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:19.672421    2578 flags.go:64] FLAG: --registry-qps="5"
Oct 30 21:52:19.678203 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:19.672426    2578 flags.go:64] FLAG: --reserved-cpus=""
Oct 30 21:52:19.678203 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:19.672430    2578 flags.go:64] FLAG: --reserved-memory=""
Oct 30 21:52:19.678203 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:19.672436    2578 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Oct 30 21:52:19.678203 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:19.672441    2578 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Oct 30 21:52:19.678203 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:19.672447    2578 flags.go:64] FLAG: --rotate-certificates="false"
Oct 30 21:52:19.678203 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:19.672452    2578 flags.go:64] FLAG: --rotate-server-certificates="false"
Oct 30 21:52:19.678203 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:19.672456    2578 flags.go:64] FLAG: --runonce="false"
Oct 30 21:52:19.678203 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:19.672461    2578 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Oct 30 21:52:19.678203 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:19.672466    2578 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Oct 30 21:52:19.678203 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:19.672471    2578 flags.go:64] FLAG: --seccomp-default="false"
Oct 30 21:52:19.678203 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:19.672475    2578 flags.go:64] FLAG: --serialize-image-pulls="true"
Oct 30 21:52:19.678203 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:19.672480    2578 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Oct 30 21:52:19.678203 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:19.672484    2578 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Oct 30 21:52:19.678203 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:19.672490    2578 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Oct 30 21:52:19.678203 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:19.672495    2578 flags.go:64] FLAG: --storage-driver-password="root"
Oct 30 21:52:19.678836 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:19.672500    2578 flags.go:64] FLAG: --storage-driver-secure="false"
Oct 30 21:52:19.678836 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:19.672505    2578 flags.go:64] FLAG: --storage-driver-table="stats"
Oct 30 21:52:19.678836 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:19.672509    2578 flags.go:64] FLAG: --storage-driver-user="root"
Oct 30 21:52:19.678836 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:19.672514    2578 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Oct 30 21:52:19.678836 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:19.672520    2578 flags.go:64] FLAG: --sync-frequency="1m0s"
Oct 30 21:52:19.678836 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:19.672526    2578 flags.go:64] FLAG: --system-cgroups=""
Oct 30 21:52:19.678836 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:19.672531    2578 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Oct 30 21:52:19.678836 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:19.672540    2578 flags.go:64] FLAG: --system-reserved-cgroup=""
Oct 30 21:52:19.678836 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:19.672545    2578 flags.go:64] FLAG: --tls-cert-file=""
Oct 30 21:52:19.678836 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:19.672549    2578 flags.go:64] FLAG: --tls-cipher-suites="[]"
Oct 30 21:52:19.678836 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:19.672555    2578 flags.go:64] FLAG: --tls-min-version=""
Oct 30 21:52:19.678836 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:19.672560    2578 flags.go:64] FLAG: --tls-private-key-file=""
Oct 30 21:52:19.678836 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:19.672564    2578 flags.go:64] FLAG: --topology-manager-policy="none"
Oct 30 21:52:19.678836 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:19.672568    2578 flags.go:64] FLAG: --topology-manager-policy-options=""
Oct 30 21:52:19.678836 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:19.672573    2578 flags.go:64] FLAG: --topology-manager-scope="container"
Oct 30 21:52:19.678836 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:19.672578    2578 flags.go:64] FLAG: --v="2"
Oct 30 21:52:19.678836 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:19.672585    2578 flags.go:64] FLAG: --version="false"
Oct 30 21:52:19.678836 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:19.672591    2578 flags.go:64] FLAG: --vmodule=""
Oct 30 21:52:19.678836 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:19.672598    2578 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Oct 30 21:52:19.678836 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:19.672603    2578 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Oct 30 21:52:19.678836 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.672828    2578 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 21:52:19.678836 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.672835    2578 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 21:52:19.678836 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.672840    2578 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 21:52:19.678836 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.672844    2578 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 21:52:19.679408 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.672848    2578 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 21:52:19.679408 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.672852    2578 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 21:52:19.679408 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.672857    2578 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 21:52:19.679408 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.672861    2578 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 21:52:19.679408 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.672865    2578 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 21:52:19.679408 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.672869    2578 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 21:52:19.679408 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.672876    2578 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 21:52:19.679408 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.672883    2578 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 21:52:19.679408 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.672887    2578 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 21:52:19.679408 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.672891    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 21:52:19.679408 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.672896    2578 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 21:52:19.679408 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.672900    2578 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 21:52:19.679408 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.672904    2578 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 21:52:19.679408 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.672912    2578 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 21:52:19.679408 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.672916    2578 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 21:52:19.679408 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.672921    2578 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 21:52:19.679408 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.672925    2578 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 21:52:19.679408 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.672929    2578 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 21:52:19.679408 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.672933    2578 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 21:52:19.679903 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.672937    2578 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 21:52:19.679903 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.672941    2578 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 21:52:19.679903 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.672946    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 21:52:19.679903 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.672950    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 21:52:19.679903 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.672954    2578 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 21:52:19.679903 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.672958    2578 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 21:52:19.679903 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.672962    2578 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 21:52:19.679903 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.672967    2578 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 21:52:19.679903 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.672971    2578 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 21:52:19.679903 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.672975    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 21:52:19.679903 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.672978    2578 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 21:52:19.679903 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.672982    2578 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 21:52:19.679903 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.672986    2578 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 21:52:19.679903 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.672990    2578 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 21:52:19.679903 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.672994    2578 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 21:52:19.679903 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.672999    2578 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 21:52:19.679903 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.673003    2578 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 21:52:19.679903 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.673007    2578 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 21:52:19.679903 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.673014    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 21:52:19.679903 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.673018    2578 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 21:52:19.680529 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.673022    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 21:52:19.680529 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.673026    2578 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 21:52:19.680529 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.673031    2578 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 21:52:19.680529 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.673035    2578 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 21:52:19.680529 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.673039    2578 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 21:52:19.680529 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.673043    2578 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 21:52:19.680529 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.673048    2578 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 21:52:19.680529 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.673054    2578 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 21:52:19.680529 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.673058    2578 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 21:52:19.680529 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.673063    2578 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 21:52:19.680529 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.673067    2578 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 21:52:19.680529 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.673071    2578 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 21:52:19.680529 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.673075    2578 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 21:52:19.680529 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.673079    2578 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 21:52:19.680529 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.673083    2578 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 21:52:19.680529 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.673087    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 21:52:19.680529 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.673092    2578 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 21:52:19.680529 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.673096    2578 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 21:52:19.680529 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.673100    2578 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 21:52:19.680958 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.673105    2578 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 21:52:19.680958 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.673109    2578 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 21:52:19.680958 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.673113    2578 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 21:52:19.680958 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.673135    2578 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 21:52:19.680958 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.673139    2578 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 21:52:19.680958 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.673144    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 21:52:19.680958 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.673148    2578 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 21:52:19.680958 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.673152    2578 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 21:52:19.680958 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.673156    2578 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 21:52:19.680958 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.673160    2578 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 21:52:19.680958 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.673164    2578 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 21:52:19.680958 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.673170    2578 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 21:52:19.680958 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.673174    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 21:52:19.680958 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.673178    2578 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 21:52:19.680958 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.673182    2578 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 21:52:19.680958 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.673186    2578 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 21:52:19.680958 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.673190    2578 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 21:52:19.680958 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.673195    2578 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 21:52:19.680958 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.673199    2578 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 21:52:19.681438 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.673205    2578 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 21:52:19.681438 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.673210    2578 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 21:52:19.681438 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.673218    2578 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 21:52:19.681438 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.673224    2578 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 21:52:19.681438 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.673229    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 21:52:19.681438 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.673233    2578 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 21:52:19.681438 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.673238    2578 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 21:52:19.681438 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.673242    2578 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 21:52:19.681438 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.673247    2578 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 21:52:19.681438 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.673251    2578 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 21:52:19.681438 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.673255    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 21:52:19.681438 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.673260    2578 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 21:52:19.681438 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.673264    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 21:52:19.681438 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.673269    2578 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 21:52:19.681438 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.673273    2578 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 21:52:19.681438 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.673277    2578 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 21:52:19.681438 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.673281    2578 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 21:52:19.681438 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.673286    2578 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 21:52:19.681438 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.673291    2578 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 21:52:19.681882 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.673296    2578 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 21:52:19.681882 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.673300    2578 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 21:52:19.681882 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.673313    2578 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 21:52:19.681882 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.673317    2578 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 21:52:19.681882 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.673322    2578 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 21:52:19.681882 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.673327    2578 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 21:52:19.681882 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:19.673374    2578 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 30 21:52:19.681882 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.673435    2578 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 21:52:19.681882 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.673443    2578 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 21:52:19.681882 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.673447    2578 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 21:52:19.681882 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.673453    2578 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 21:52:19.681882 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.673460    2578 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 21:52:19.681882 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.673467    2578 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 21:52:19.681882 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.673475    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 21:52:19.681882 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.673479    2578 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 21:52:19.681882 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.673484    2578 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 21:52:19.682279 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.673489    2578 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 21:52:19.682279 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.673493    2578 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 21:52:19.682279 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.673497    2578 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 21:52:19.682279 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.673502    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 21:52:19.682279 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.673507    2578 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 21:52:19.682279 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.673511    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 21:52:19.682279 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.673515    2578 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 21:52:19.682279 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.673519    2578 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 21:52:19.682279 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.673524    2578 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 21:52:19.682279 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.673528    2578 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 21:52:19.682279 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.673532    2578 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 21:52:19.682279 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.673536    2578 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 21:52:19.682279 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.673540    2578 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 21:52:19.682279 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.673545    2578 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 21:52:19.682279 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.673549    2578 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 21:52:19.682279 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.673553    2578 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 21:52:19.682279 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.673558    2578 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 21:52:19.682279 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.673562    2578 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 21:52:19.682279 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.673566    2578 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 21:52:19.682279 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.673570    2578 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 21:52:19.682738 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.673574    2578 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 21:52:19.682738 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.673578    2578 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 21:52:19.682738 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.673584    2578 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 21:52:19.682738 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.673588    2578 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 21:52:19.682738 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.673593    2578 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 21:52:19.682738 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.673597    2578 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 21:52:19.682738 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.673601    2578 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 21:52:19.682738 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.673605    2578 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 21:52:19.682738 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.673612    2578 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 21:52:19.682738 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.673618    2578 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 21:52:19.682738 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.673623    2578 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 21:52:19.682738 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.673627    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 21:52:19.682738 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.673632    2578 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 21:52:19.682738 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.673646    2578 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 21:52:19.682738 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.673651    2578 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 21:52:19.682738 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.673655    2578 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 21:52:19.682738 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.673660    2578 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 21:52:19.682738 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.673665    2578 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 21:52:19.682738 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.673669    2578 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 21:52:19.683204 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.673674    2578 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 21:52:19.683204 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.673678    2578 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 21:52:19.683204 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.673682    2578 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 21:52:19.683204 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.673686    2578 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 21:52:19.683204 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.673690    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 21:52:19.683204 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.673695    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 21:52:19.683204 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.673699    2578 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 21:52:19.683204 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.673703    2578 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 21:52:19.683204 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.673707    2578 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 21:52:19.683204 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.673712    2578 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 21:52:19.683204 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.673716    2578 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 21:52:19.683204 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.673720    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 21:52:19.683204 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.673724    2578 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 21:52:19.683204 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.673729    2578 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 21:52:19.683204 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.673733    2578 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 21:52:19.683204 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.673739    2578 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 21:52:19.683204 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.673743    2578 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 21:52:19.683204 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.673748    2578 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 21:52:19.683204 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.673752    2578 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 21:52:19.683204 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.673756    2578 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 21:52:19.683660 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.673761    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 21:52:19.683660 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.673766    2578 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 21:52:19.683660 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.673771    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 21:52:19.683660 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.673776    2578 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 21:52:19.683660 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.673780    2578 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 21:52:19.683660 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.673784    2578 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 21:52:19.683660 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.673788    2578 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 21:52:19.683660 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.673794    2578 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 21:52:19.683660 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.673798    2578 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 21:52:19.683660 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.673802    2578 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 21:52:19.683660 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.673806    2578 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 21:52:19.683660 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.673810    2578 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 21:52:19.683660 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.673815    2578 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 21:52:19.683660 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.673819    2578 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 21:52:19.683660 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.673823    2578 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 21:52:19.683660 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.673827    2578 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 21:52:19.683660 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.673832    2578 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 21:52:19.683660 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.673836    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 21:52:19.683660 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.673841    2578 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 21:52:19.684098 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.673845    2578 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 21:52:19.684098 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.673849    2578 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 21:52:19.684098 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.673853    2578 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 21:52:19.684098 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.673857    2578 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 21:52:19.684098 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.673862    2578 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 21:52:19.684098 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.673866    2578 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 21:52:19.684098 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.673870    2578 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 21:52:19.684098 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.673874    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 21:52:19.684098 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.673879    2578 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 21:52:19.684098 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.673883    2578 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 21:52:19.684098 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.673887    2578 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 21:52:19.684098 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.673891    2578 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 21:52:19.684098 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.673895    2578 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 21:52:19.684098 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.673899    2578 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 21:52:19.684098 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.673904    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 21:52:19.684098 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.673910    2578 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 21:52:19.684098 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.673917    2578 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 21:52:19.684098 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.673921    2578 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 21:52:19.684098 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.673925    2578 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 21:52:19.684587 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:19.681890    2578 server.go:525] "Kubelet version" kubeletVersion="v1.35.9"
Oct 30 21:52:19.684587 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:19.681907    2578 server.go:527] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Oct 30 21:52:19.684587 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:19.681924    2578 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Oct 30 21:52:19.684587 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:19.681929    2578 watchdog_linux.go:138] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Oct 30 21:52:19.684587 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.681984    2578 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 21:52:19.684587 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.681989    2578 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 21:52:19.684587 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.681993    2578 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 21:52:19.684587 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.681996    2578 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 21:52:19.684587 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.681999    2578 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 21:52:19.684587 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.682002    2578 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 21:52:19.684587 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.682005    2578 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 21:52:19.684587 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.682007    2578 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 21:52:19.684587 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.682010    2578 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 21:52:19.684587 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.682012    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 21:52:19.684587 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.682015    2578 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 21:52:19.684587 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.682017    2578 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 21:52:19.684587 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.682020    2578 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 21:52:19.684587 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.682022    2578 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 21:52:19.684587 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.682025    2578 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 21:52:19.685023 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.682027    2578 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 21:52:19.685023 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.682030    2578 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 21:52:19.685023 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.682032    2578 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 21:52:19.685023 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.682035    2578 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 21:52:19.685023 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.682037    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 21:52:19.685023 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.682040    2578 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 21:52:19.685023 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.682042    2578 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 21:52:19.685023 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.682045    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 21:52:19.685023 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.682047    2578 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 21:52:19.685023 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.682050    2578 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 21:52:19.685023 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.682053    2578 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 21:52:19.685023 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.682055    2578 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 21:52:19.685023 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.682058    2578 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 21:52:19.685023 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.682060    2578 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 21:52:19.685023 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.682064    2578 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 21:52:19.685023 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.682067    2578 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 21:52:19.685023 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.682069    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 21:52:19.685023 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.682072    2578 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 21:52:19.685023 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.682075    2578 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 21:52:19.685023 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.682077    2578 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 21:52:19.685512 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.682080    2578 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 21:52:19.685512 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.682084    2578 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 21:52:19.685512 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.682087    2578 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 21:52:19.685512 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.682090    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 21:52:19.685512 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.682093    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 21:52:19.685512 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.682096    2578 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 21:52:19.685512 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.682098    2578 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 21:52:19.685512 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.682102    2578 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 21:52:19.685512 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.682106    2578 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 21:52:19.685512 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.682109    2578 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 21:52:19.685512 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.682111    2578 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 21:52:19.685512 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.682114    2578 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 21:52:19.685512 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.682135    2578 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 21:52:19.685512 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.682138    2578 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 21:52:19.685512 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.682141    2578 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 21:52:19.685512 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.682143    2578 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 21:52:19.685512 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.682146    2578 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 21:52:19.685512 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.682149    2578 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 21:52:19.685512 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.682151    2578 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 21:52:19.685955 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.682154    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 21:52:19.685955 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.682156    2578 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 21:52:19.685955 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.682159    2578 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 21:52:19.685955 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.682161    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 21:52:19.685955 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.682164    2578 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 21:52:19.685955 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.682166    2578 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 21:52:19.685955 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.682169    2578 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 21:52:19.685955 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.682173    2578 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 21:52:19.685955 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.682176    2578 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 21:52:19.685955 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.682179    2578 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 21:52:19.685955 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.682182    2578 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 21:52:19.685955 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.682185    2578 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 21:52:19.685955 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.682187    2578 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 21:52:19.685955 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.682190    2578 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 21:52:19.685955 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.682192    2578 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 21:52:19.685955 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.682195    2578 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 21:52:19.685955 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.682197    2578 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 21:52:19.685955 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.682200    2578 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 21:52:19.685955 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.682202    2578 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 21:52:19.686439 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.682205    2578 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 21:52:19.686439 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.682207    2578 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 21:52:19.686439 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.682210    2578 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 21:52:19.686439 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.682213    2578 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 21:52:19.686439 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.682216    2578 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 21:52:19.686439 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.682218    2578 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 21:52:19.686439 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.682220    2578 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 21:52:19.686439 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.682223    2578 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 21:52:19.686439 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.682226    2578 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 21:52:19.686439 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.682228    2578 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 21:52:19.686439 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.682231    2578 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 21:52:19.686439 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.682233    2578 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 21:52:19.686439 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.682235    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 21:52:19.686439 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.682238    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 21:52:19.686439 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.682240    2578 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 21:52:19.686439 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.682242    2578 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 21:52:19.686439 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.682245    2578 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 21:52:19.686439 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.682247    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 21:52:19.686439 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.682250    2578 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 21:52:19.686439 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.682253    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 21:52:19.686905 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.682256    2578 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 21:52:19.686905 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.682258    2578 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 21:52:19.686905 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.682261    2578 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 21:52:19.686905 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.682264    2578 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 21:52:19.686905 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.682267    2578 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 21:52:19.686905 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.682269    2578 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 21:52:19.686905 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.682271    2578 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 21:52:19.686905 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.682274    2578 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 21:52:19.686905 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.682277    2578 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 21:52:19.686905 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.682279    2578 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 21:52:19.686905 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.682282    2578 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 21:52:19.686905 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.682285    2578 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 21:52:19.686905 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.682287    2578 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 21:52:19.686905 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:19.682314    2578 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 30 21:52:19.686905 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.682804    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 21:52:19.686905 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.682809    2578 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 21:52:19.687306 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.682812    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 21:52:19.687306 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.682815    2578 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 21:52:19.687306 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.682817    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 21:52:19.687306 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.682820    2578 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 21:52:19.687306 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.682822    2578 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 21:52:19.687306 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.682825    2578 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 21:52:19.687306 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.682827    2578 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 21:52:19.687306 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.682830    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 21:52:19.687306 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.682832    2578 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 21:52:19.687306 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.682835    2578 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 21:52:19.687306 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.682837    2578 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 21:52:19.687306 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.682840    2578 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 21:52:19.687306 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.682842    2578 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 21:52:19.687306 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.682844    2578 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 21:52:19.687306 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.682847    2578 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 21:52:19.687306 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.682849    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 21:52:19.687306 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.682852    2578 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 21:52:19.687306 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.682855    2578 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 21:52:19.687306 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.682858    2578 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 21:52:19.687306 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.682860    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 21:52:19.687788 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.682863    2578 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 21:52:19.687788 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.682865    2578 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 21:52:19.687788 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.682868    2578 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 21:52:19.687788 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.682870    2578 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 21:52:19.687788 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.682873    2578 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 21:52:19.687788 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.682875    2578 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 21:52:19.687788 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.682877    2578 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 21:52:19.687788 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.682880    2578 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 21:52:19.687788 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.682883    2578 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 21:52:19.687788 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.682885    2578 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 21:52:19.687788 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.682888    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 21:52:19.687788 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.682890    2578 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 21:52:19.687788 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.682892    2578 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 21:52:19.687788 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.682895    2578 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 21:52:19.687788 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.682897    2578 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 21:52:19.687788 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.682900    2578 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 21:52:19.687788 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.682902    2578 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 21:52:19.687788 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.682905    2578 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 21:52:19.687788 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.682907    2578 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 21:52:19.687788 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.682909    2578 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 21:52:19.688271 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.682912    2578 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 21:52:19.688271 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.682914    2578 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 21:52:19.688271 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.682916    2578 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 21:52:19.688271 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.682919    2578 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 21:52:19.688271 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.682921    2578 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 21:52:19.688271 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.682924    2578 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 21:52:19.688271 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.682926    2578 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 21:52:19.688271 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.682929    2578 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 21:52:19.688271 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.682932    2578 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 21:52:19.688271 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.682934    2578 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 21:52:19.688271 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.682936    2578 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 21:52:19.688271 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.682939    2578 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 21:52:19.688271 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.682941    2578 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 21:52:19.688271 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.682944    2578 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 21:52:19.688271 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.682946    2578 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 21:52:19.688271 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.682949    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 21:52:19.688271 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.682951    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 21:52:19.688271 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.682953    2578 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 21:52:19.688271 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.682956    2578 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 21:52:19.688271 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.682958    2578 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 21:52:19.688731 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.682960    2578 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 21:52:19.688731 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.682963    2578 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 21:52:19.688731 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.682965    2578 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 21:52:19.688731 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.682967    2578 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 21:52:19.688731 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.682970    2578 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 21:52:19.688731 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.682972    2578 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 21:52:19.688731 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.682975    2578 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 21:52:19.688731 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.682977    2578 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 21:52:19.688731 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.682981    2578 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 21:52:19.688731 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.682984    2578 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 21:52:19.688731 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.682987    2578 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 21:52:19.688731 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.682989    2578 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 21:52:19.688731 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.682992    2578 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 21:52:19.688731 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.682994    2578 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 21:52:19.688731 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.682997    2578 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 21:52:19.688731 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.683000    2578 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 21:52:19.688731 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.683004    2578 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 21:52:19.688731 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.683006    2578 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 21:52:19.688731 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.683009    2578 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 21:52:19.689201 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.683012    2578 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 21:52:19.689201 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.683014    2578 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 21:52:19.689201 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.683017    2578 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 21:52:19.689201 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.683019    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 21:52:19.689201 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.683022    2578 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 21:52:19.689201 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.683024    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 21:52:19.689201 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.683027    2578 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 21:52:19.689201 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.683030    2578 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 21:52:19.689201 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.683032    2578 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 21:52:19.689201 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.683034    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 21:52:19.689201 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.683037    2578 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 21:52:19.689201 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.683039    2578 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 21:52:19.689201 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.683042    2578 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 21:52:19.689201 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.683045    2578 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 21:52:19.689201 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.683047    2578 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 21:52:19.689201 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.683050    2578 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 21:52:19.689201 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.683052    2578 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 21:52:19.689201 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.683054    2578 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 21:52:19.689201 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.683057    2578 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 21:52:19.689647 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.683059    2578 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 21:52:19.689647 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.683062    2578 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 21:52:19.689647 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.683064    2578 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 21:52:19.689647 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.683067    2578 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 21:52:19.689647 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.683069    2578 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 21:52:19.689647 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.683072    2578 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 21:52:19.689647 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:19.683094    2578 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 30 21:52:19.689647 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.683140    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 21:52:19.689647 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.683144    2578 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 21:52:19.689647 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.683148    2578 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 21:52:19.689647 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.683151    2578 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 21:52:19.689647 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.683154    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 21:52:19.689647 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.683157    2578 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 21:52:19.689647 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.683161    2578 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 21:52:19.689647 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.683163    2578 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 21:52:19.689647 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.683166    2578 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 21:52:19.690017 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.683168    2578 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 21:52:19.690017 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.683171    2578 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 21:52:19.690017 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.683173    2578 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 21:52:19.690017 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.683176    2578 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 21:52:19.690017 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.683179    2578 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 21:52:19.690017 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.683181    2578 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 21:52:19.690017 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.683184    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 21:52:19.690017 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.683186    2578 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 21:52:19.690017 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.683189    2578 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 21:52:19.690017 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.683191    2578 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 21:52:19.690017 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.683193    2578 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 21:52:19.690017 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.683196    2578 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 21:52:19.690017 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.683198    2578 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 21:52:19.690017 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.683200    2578 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 21:52:19.690017 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.683203    2578 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 21:52:19.690017 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.683205    2578 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 21:52:19.690017 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.683208    2578 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 21:52:19.690017 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.683210    2578 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 21:52:19.690017 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.683213    2578 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 21:52:19.690017 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.683216    2578 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 21:52:19.690581 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.683218    2578 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 21:52:19.690581 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.683221    2578 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 21:52:19.690581 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.683224    2578 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 21:52:19.690581 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.683226    2578 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 21:52:19.690581 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.683228    2578 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 21:52:19.690581 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.683231    2578 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 21:52:19.690581 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.683233    2578 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 21:52:19.690581 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.683236    2578 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 21:52:19.690581 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.683238    2578 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 21:52:19.690581 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.683241    2578 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 21:52:19.690581 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.683243    2578 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 21:52:19.690581 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.683245    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 21:52:19.690581 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.683248    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 21:52:19.690581 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.683250    2578 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 21:52:19.690581 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.683253    2578 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 21:52:19.690581 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.683255    2578 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 21:52:19.690581 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.683258    2578 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 21:52:19.690581 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.683261    2578 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 21:52:19.690581 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.683263    2578 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 21:52:19.690581 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.683265    2578 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 21:52:19.691066 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.683268    2578 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 21:52:19.691066 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.683270    2578 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 21:52:19.691066 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.683273    2578 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 21:52:19.691066 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.683275    2578 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 21:52:19.691066 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.683278    2578 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 21:52:19.691066 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.683282    2578 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 21:52:19.691066 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.683285    2578 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 21:52:19.691066 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.683287    2578 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 21:52:19.691066 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.683290    2578 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 21:52:19.691066 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.683293    2578 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 21:52:19.691066 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.683295    2578 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 21:52:19.691066 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.683299    2578 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 21:52:19.691066 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.683302    2578 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 21:52:19.691066 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.683306    2578 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 21:52:19.691066 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.683309    2578 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 21:52:19.691066 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.683312    2578 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 21:52:19.691066 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.683315    2578 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 21:52:19.691066 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.683318    2578 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 21:52:19.691066 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.683320    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 21:52:19.691527 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.683323    2578 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 21:52:19.691527 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.683325    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 21:52:19.691527 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.683328    2578 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 21:52:19.691527 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.683330    2578 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 21:52:19.691527 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.683332    2578 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 21:52:19.691527 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.683335    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 21:52:19.691527 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.683337    2578 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 21:52:19.691527 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.683340    2578 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 21:52:19.691527 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.683343    2578 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 21:52:19.691527 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.683346    2578 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 21:52:19.691527 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.683348    2578 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 21:52:19.691527 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.683351    2578 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 21:52:19.691527 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.683353    2578 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 21:52:19.691527 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.683356    2578 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 21:52:19.691527 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.683358    2578 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 21:52:19.691527 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.683361    2578 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 21:52:19.691527 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.683363    2578 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 21:52:19.691527 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.683366    2578 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 21:52:19.691527 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.683368    2578 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 21:52:19.691527 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.683371    2578 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 21:52:19.691988 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.683373    2578 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 21:52:19.691988 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.683375    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 21:52:19.691988 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.683378    2578 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 21:52:19.691988 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.683381    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 21:52:19.691988 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.683383    2578 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 21:52:19.691988 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.683385    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 21:52:19.691988 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.683388    2578 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 21:52:19.691988 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.683390    2578 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 21:52:19.691988 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.683393    2578 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 21:52:19.691988 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.683396    2578 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 21:52:19.691988 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.683398    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 21:52:19.691988 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.683401    2578 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 21:52:19.691988 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.683403    2578 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 21:52:19.691988 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.683406    2578 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 21:52:19.691988 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.683408    2578 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 21:52:19.691988 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.683411    2578 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 21:52:19.691988 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.683413    2578 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 21:52:19.691988 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:19.683415    2578 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 21:52:19.691988 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:19.683484    2578 server.go:957] "Client rotation is on, will bootstrap in background"
Oct 30 21:52:19.692432 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:19.686348    2578 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Oct 30 21:52:19.692432 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:19.687285    2578 server.go:1014] "Starting client certificate rotation"
Oct 30 21:52:19.692432 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:19.687378    2578 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Oct 30 21:52:19.692432 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:19.687411    2578 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Oct 30 21:52:19.692432 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:19.689327    2578 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Oct 30 21:52:19.698020 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:19.698004    2578 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Oct 30 21:52:19.708033 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:19.708015    2578 log.go:25] "Validated CRI v1 runtime API"
Oct 30 21:52:19.713801 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:19.713786    2578 log.go:25] "Validated CRI v1 image API"
Oct 30 21:52:19.714654 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:19.714620    2578 server.go:1424] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Oct 30 21:52:19.719591 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:19.719572    2578 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 807d4c8b-be93-4a35-b656-6e19d4292167:/dev/nvme0n1p3 96540798-6f1c-4a39-a1ec-8fe598f5180f:/dev/nvme0n1p4]
Oct 30 21:52:19.719638 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:19.719590    2578 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:24 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:26 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:32 fsType:tmpfs blockSize:0} composefs_0-34:{mountpoint:/ major:0 minor:34 fsType:overlay blockSize:0}]
Oct 30 21:52:19.724755 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:19.724642    2578 manager.go:217] Machine: {Timestamp:2026-10-30 21:52:19.723272371 +0000 UTC m=+0.362972584 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3273797 MemoryCapacity:32812015616 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2b39ebd184acc12d51b4696614397b SystemUUID:ec2b39eb-d184-acc1-2d51-b4696614397b BootID:ce4b67e4-503b-437c-8dbe-13e77d47e3f4 Filesystems:[{Device:/run DeviceMajor:0 DeviceMinor:26 Capacity:6562406400 Type:vfs Inodes:819200 HasInodes:true} {Device:composefs_0-34 DeviceMajor:0 DeviceMinor:34 Capacity:6422528 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:32 Capacity:16406007808 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:24 Capacity:16406007808 Type:vfs Inodes:4005373 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:48:66:66:c1:e7 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:48:66:66:c1:e7 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:ee:6b:d9:0c:87:57 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812015616 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Oct 30 21:52:19.725387 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:19.725376    2578 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Oct 30 21:52:19.725472 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:19.725460    2578 manager.go:233] Version: {KernelVersion:5.14.0-687.60.1.el9_8.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.8.20261023-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Oct 30 21:52:19.727971 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:19.727950    2578 container_manager_linux.go:275] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Oct 30 21:52:19.728111 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:19.727973    2578 container_manager_linux.go:280] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-131-202.us-west-1.compute.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"/system.slice","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{},"system-reserved-compressible":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Oct 30 21:52:19.728715 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:19.728705    2578 topology_manager.go:143] "Creating topology manager with none policy"
Oct 30 21:52:19.728749 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:19.728717    2578 container_manager_linux.go:311] "Creating device plugin manager"
Oct 30 21:52:19.728749 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:19.728744    2578 manager.go:144] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Oct 30 21:52:19.728799 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:19.728757    2578 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Oct 30 21:52:19.729773 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:19.729763    2578 container_manager_linux.go:320] "Creating Dynamic Resource Allocation (DRA) manager"
Oct 30 21:52:19.734077 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:19.734065    2578 state_mem.go:41] "Initialized" logger="CPUManager state memory"
Oct 30 21:52:19.734223 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:19.734212    2578 server.go:1235] "Using root directory" path="/var/lib/kubelet"
Oct 30 21:52:19.735031 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:19.735021    2578 kubelet.go:493] "Attempting to sync node with API server"
Oct 30 21:52:19.735067 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:19.735035    2578 kubelet.go:394] "Adding static pod path" path="/etc/kubernetes/manifests"
Oct 30 21:52:19.735067 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:19.735044    2578 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Oct 30 21:52:19.735067 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:19.735053    2578 kubelet.go:405] "Adding apiserver pod source"
Oct 30 21:52:19.735067 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:19.735062    2578 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Oct 30 21:52:19.735461 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:19.735446    2578 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Oct 30 21:52:19.737700 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:19.737682    2578 kuberuntime_manager.go:304] "Container runtime initialized" containerRuntime="cri-o" version="1.35.10-3.rhaos4.22.git9383502.el9" apiVersion="v1"
Oct 30 21:52:19.744805 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:19.744789    2578 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Oct 30 21:52:19.745697 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:19.745684    2578 kubelet.go:963] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Oct 30 21:52:19.745748 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:19.745714    2578 kubelet.go:990] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Oct 30 21:52:19.745862 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:19.745852    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Oct 30 21:52:19.745893 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:19.745868    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Oct 30 21:52:19.745893 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:19.745874    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Oct 30 21:52:19.745893 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:19.745880    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Oct 30 21:52:19.745893 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:19.745886    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Oct 30 21:52:19.745893 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:19.745891    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Oct 30 21:52:19.746591 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:19.746582    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Oct 30 21:52:19.746621 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:19.746592    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Oct 30 21:52:19.746621 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:19.746599    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Oct 30 21:52:19.746621 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:19.746605    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Oct 30 21:52:19.746621 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:19.746617    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Oct 30 21:52:19.746741 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:19.746626    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Oct 30 21:52:19.747369 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:19.747360    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Oct 30 21:52:19.747369 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:19.747369    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Oct 30 21:52:19.751286 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:19.751273    2578 server.go:1263] "Started kubelet"
Oct 30 21:52:19.751345 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:19.751314    2578 server.go:182] "Starting to listen" address="0.0.0.0" port=10250
Oct 30 21:52:19.751410 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:19.751359    2578 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Oct 30 21:52:19.751520 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:19.751428    2578 server_v1.go:49] "podresources" method="list" useActivePods=true
Oct 30 21:52:19.752033 ip-10-0-131-202 systemd[1]: Started Kubernetes Kubelet.
Oct 30 21:52:19.754044 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:19.754024    2578 server.go:254] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Oct 30 21:52:19.754044 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:19.754037    2578 server.go:317] "Adding debug handlers to kubelet server"
Oct 30 21:52:19.756728 ip-10-0-131-202 kubenswrapper[2578]: E1030 21:52:19.756712    2578 kubelet.go:1679] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Oct 30 21:52:19.759838 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:19.759820    2578 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Oct 30 21:52:19.761173 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:19.761159    2578 fs_resource_analyzer.go:69] "Starting FS ResourceAnalyzer"
Oct 30 21:52:19.761290 ip-10-0-131-202 kubenswrapper[2578]: E1030 21:52:19.761270    2578 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-131-202.us-west-1.compute.internal\" not found"
Oct 30 21:52:19.761974 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:19.761924    2578 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Oct 30 21:52:19.761974 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:19.761925    2578 volume_manager.go:309] "The desired_state_of_world populator starts"
Oct 30 21:52:19.761974 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:19.761944    2578 volume_manager.go:311] "Starting Kubelet Volume Manager"
Oct 30 21:52:19.761974 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:19.762028    2578 reconstruct.go:97] "Volume reconstruction finished"
Oct 30 21:52:19.761974 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:19.762038    2578 reconciler.go:29] "Reconciler: start to sync state"
Oct 30 21:52:19.762333 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:19.762222    2578 factory.go:153] Registering CRI-O factory
Oct 30 21:52:19.762333 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:19.762237    2578 factory.go:223] Registration of the crio container factory successfully
Oct 30 21:52:19.762333 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:19.762286    2578 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Oct 30 21:52:19.762333 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:19.762296    2578 factory.go:55] Registering systemd factory
Oct 30 21:52:19.762333 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:19.762303    2578 factory.go:223] Registration of the systemd container factory successfully
Oct 30 21:52:19.762333 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:19.762315    2578 factory.go:103] Registering Raw factory
Oct 30 21:52:19.762333 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:19.762327    2578 manager.go:1199] Started watching for new ooms in manager
Oct 30 21:52:19.763345 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:19.763330    2578 manager.go:319] Starting recovery of all containers
Oct 30 21:52:19.774658 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:19.774642    2578 manager.go:324] Recovery completed
Oct 30 21:52:19.778424 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:19.778410    2578 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 21:52:19.780061 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:19.780033    2578 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-131-202.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Oct 30 21:52:19.780108 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:19.780081    2578 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-131-202.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 21:52:19.780108 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:19.780097    2578 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-131-202.us-west-1.compute.internal" event="NodeHasSufficientPID"
Oct 30 21:52:19.780620 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:19.780607    2578 cpu_manager.go:226] "Starting" policy="none"
Oct 30 21:52:19.780620 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:19.780617    2578 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 21:52:19.780763 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:19.780617    2578 cpu_manager.go:227] "Reconciling" reconcilePeriod="10s"
Oct 30 21:52:19.780763 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:19.780647    2578 state_mem.go:41] "Initialized" logger="CPUManager state checkpoint.CPUManager state memory"
Oct 30 21:52:19.781286 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:19.781271    2578 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-131-202.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Oct 30 21:52:19.781362 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:19.781301    2578 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-131-202.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 21:52:19.781362 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:19.781324    2578 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-131-202.us-west-1.compute.internal" event="NodeHasSufficientPID"
Oct 30 21:52:19.782803 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:19.782791    2578 policy_none.go:50] "Start"
Oct 30 21:52:19.782855 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:19.782806    2578 memory_manager.go:187] "Starting memorymanager" policy="None"
Oct 30 21:52:19.782855 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:19.782818    2578 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Oct 30 21:52:19.784811 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:19.784800    2578 policy_none.go:44] "Start"
Oct 30 21:52:19.829831 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:19.829683    2578 node_container_manager_linux.go:130] "Enforcing system reserved compressible on cgroup" cgroupName="/system.slice" limits={"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"}
Oct 30 21:52:19.833541 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:19.833528    2578 manager.go:341] "Starting Device Plugin manager"
Oct 30 21:52:19.833618 ip-10-0-131-202 kubenswrapper[2578]: E1030 21:52:19.833558    2578 manager.go:525] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Oct 30 21:52:19.833618 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:19.833566    2578 server.go:87] "Starting device plugin registration server"
Oct 30 21:52:19.833796 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:19.833783    2578 eviction_manager.go:194] "Eviction manager: starting control loop"
Oct 30 21:52:19.833841 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:19.833798    2578 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Oct 30 21:52:19.833931 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:19.833909    2578 plugin_watcher.go:53] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Oct 30 21:52:19.834477 ip-10-0-131-202 kubenswrapper[2578]: E1030 21:52:19.834454    2578 eviction_manager.go:272] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Oct 30 21:52:19.834566 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:19.834490    2578 plugin_manager.go:119] "The desired_state_of_world populator (plugin watcher) starts"
Oct 30 21:52:19.834566 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:19.834497    2578 plugin_manager.go:121] "Starting Kubelet Plugin Manager"
Oct 30 21:52:19.834566 ip-10-0-131-202 kubenswrapper[2578]: E1030 21:52:19.834511    2578 eviction_manager.go:297] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-131-202.us-west-1.compute.internal\" not found"
Oct 30 21:52:19.898132 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:19.898098    2578 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-131-202.us-west-1.compute.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Oct 30 21:52:19.908711 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:19.908659    2578 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Oct 30 21:52:19.909913 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:19.909901    2578 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Oct 30 21:52:19.909979 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:19.909919    2578 status_manager.go:249] "Starting to sync pod status with apiserver"
Oct 30 21:52:19.909979 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:19.909935    2578 kubelet.go:2521] "Starting kubelet main sync loop"
Oct 30 21:52:19.909979 ip-10-0-131-202 kubenswrapper[2578]: E1030 21:52:19.909966    2578 kubelet.go:2545] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Oct 30 21:52:19.934333 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:19.934302    2578 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 21:52:19.935009 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:19.934993    2578 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-131-202.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Oct 30 21:52:19.935065 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:19.935024    2578 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-131-202.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 21:52:19.935065 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:19.935034    2578 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-131-202.us-west-1.compute.internal" event="NodeHasSufficientPID"
Oct 30 21:52:19.935065 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:19.935057    2578 kubelet_node_status.go:77] "Attempting to register node" node="ip-10-0-131-202.us-west-1.compute.internal"
Oct 30 21:52:19.946510 ip-10-0-131-202 kubenswrapper[2578]: E1030 21:52:19.946489    2578 controller.go:201] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-131-202.us-west-1.compute.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Oct 30 21:52:19.947185 ip-10-0-131-202 kubenswrapper[2578]: E1030 21:52:19.946483    2578 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-131-202.us-west-1.compute.internal.18e36d32601094ed  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-131-202.us-west-1.compute.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-131-202.us-west-1.compute.internal,},FirstTimestamp:2026-10-30 21:52:19.751081197 +0000 UTC m=+0.390781411,LastTimestamp:2026-10-30 21:52:19.751081197 +0000 UTC m=+0.390781411,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-131-202.us-west-1.compute.internal,}"
Oct 30 21:52:19.965371 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:19.965351    2578 reflector.go:446] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Oct 30 21:52:19.965438 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:19.965404    2578 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-nhlqx"
Oct 30 21:52:20.010647 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:20.010618    2578 kubelet.go:2608] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-202.us-west-1.compute.internal","kube-system/kube-apiserver-proxy-ip-10-0-131-202.us-west-1.compute.internal"]
Oct 30 21:52:20.010732 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:20.010676    2578 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 21:52:20.011439 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:20.011425    2578 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-131-202.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Oct 30 21:52:20.011494 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:20.011451    2578 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-131-202.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 21:52:20.011494 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:20.011461    2578 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-131-202.us-west-1.compute.internal" event="NodeHasSufficientPID"
Oct 30 21:52:20.013611 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:20.013599    2578 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 21:52:20.013766 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:20.013752    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-202.us-west-1.compute.internal"
Oct 30 21:52:20.013821 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:20.013811    2578 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 21:52:20.014314 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:20.014300    2578 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-131-202.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Oct 30 21:52:20.014382 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:20.014325    2578 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-131-202.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 21:52:20.014382 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:20.014336    2578 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-131-202.us-west-1.compute.internal" event="NodeHasSufficientPID"
Oct 30 21:52:20.014670 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:20.014657    2578 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-131-202.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Oct 30 21:52:20.014735 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:20.014685    2578 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-131-202.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 21:52:20.014735 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:20.014695    2578 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-131-202.us-west-1.compute.internal" event="NodeHasSufficientPID"
Oct 30 21:52:20.016800 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:20.016785    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-202.us-west-1.compute.internal"
Oct 30 21:52:20.016877 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:20.016809    2578 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 21:52:20.017563 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:20.017540    2578 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-131-202.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Oct 30 21:52:20.017647 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:20.017569    2578 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-131-202.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 21:52:20.017647 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:20.017583    2578 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-131-202.us-west-1.compute.internal" event="NodeHasSufficientPID"
Oct 30 21:52:20.046812 ip-10-0-131-202 kubenswrapper[2578]: E1030 21:52:20.046790    2578 kubelet.go:3364] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-131-202.us-west-1.compute.internal\" not found" node="ip-10-0-131-202.us-west-1.compute.internal"
Oct 30 21:52:20.049299 ip-10-0-131-202 kubenswrapper[2578]: E1030 21:52:20.049284    2578 kubelet.go:3364] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-131-202.us-west-1.compute.internal\" not found" node="ip-10-0-131-202.us-west-1.compute.internal"
Oct 30 21:52:20.057844 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:20.057828    2578 reflector.go:446] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 21:52:20.068953 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:20.064142    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/ec1e59c1a0a508a7c75743ab25eff8ba-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-131-202.us-west-1.compute.internal\" (UID: \"ec1e59c1a0a508a7c75743ab25eff8ba\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-202.us-west-1.compute.internal"
Oct 30 21:52:20.068953 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:20.064168    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ec1e59c1a0a508a7c75743ab25eff8ba-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-131-202.us-west-1.compute.internal\" (UID: \"ec1e59c1a0a508a7c75743ab25eff8ba\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-202.us-west-1.compute.internal"
Oct 30 21:52:20.068953 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:20.064186    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/04337a6f3666f0af07c3d2b702c6e379-config\") pod \"kube-apiserver-proxy-ip-10-0-131-202.us-west-1.compute.internal\" (UID: \"04337a6f3666f0af07c3d2b702c6e379\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-131-202.us-west-1.compute.internal"
Oct 30 21:52:20.107697 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:20.107675    2578 reflector.go:446] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 21:52:20.158041 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:20.158015    2578 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 21:52:20.158864 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:20.158824    2578 reflector.go:446] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 21:52:20.159008 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:20.158985    2578 kubelet_node_status.go:80] "Successfully registered node" node="ip-10-0-131-202.us-west-1.compute.internal"
Oct 30 21:52:20.159008 ip-10-0-131-202 kubenswrapper[2578]: E1030 21:52:20.159005    2578 kubelet_node_status.go:524] "Error updating node status, will retry" err="error getting node \"ip-10-0-131-202.us-west-1.compute.internal\": node \"ip-10-0-131-202.us-west-1.compute.internal\" not found"
Oct 30 21:52:20.162239 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:20.162207    2578 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-202.us-west-1.compute.internal"
Oct 30 21:52:20.164398 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:20.164383    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/ec1e59c1a0a508a7c75743ab25eff8ba-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-131-202.us-west-1.compute.internal\" (UID: \"ec1e59c1a0a508a7c75743ab25eff8ba\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-202.us-west-1.compute.internal"
Oct 30 21:52:20.164460 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:20.164406    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ec1e59c1a0a508a7c75743ab25eff8ba-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-131-202.us-west-1.compute.internal\" (UID: \"ec1e59c1a0a508a7c75743ab25eff8ba\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-202.us-west-1.compute.internal"
Oct 30 21:52:20.164460 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:20.164424    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/04337a6f3666f0af07c3d2b702c6e379-config\") pod \"kube-apiserver-proxy-ip-10-0-131-202.us-west-1.compute.internal\" (UID: \"04337a6f3666f0af07c3d2b702c6e379\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-131-202.us-west-1.compute.internal"
Oct 30 21:52:20.164524 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:20.164476    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/04337a6f3666f0af07c3d2b702c6e379-config\") pod \"kube-apiserver-proxy-ip-10-0-131-202.us-west-1.compute.internal\" (UID: \"04337a6f3666f0af07c3d2b702c6e379\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-131-202.us-west-1.compute.internal"
Oct 30 21:52:20.164524 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:20.164494    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ec1e59c1a0a508a7c75743ab25eff8ba-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-131-202.us-west-1.compute.internal\" (UID: \"ec1e59c1a0a508a7c75743ab25eff8ba\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-202.us-west-1.compute.internal"
Oct 30 21:52:20.164524 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:20.164494    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/ec1e59c1a0a508a7c75743ab25eff8ba-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-131-202.us-west-1.compute.internal\" (UID: \"ec1e59c1a0a508a7c75743ab25eff8ba\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-202.us-west-1.compute.internal"
Oct 30 21:52:20.225595 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:20.225577    2578 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 30 21:52:20.226198 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:20.226187    2578 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-202.us-west-1.compute.internal"
Oct 30 21:52:20.286221 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:20.286203    2578 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 30 21:52:20.349981 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:20.349962    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-202.us-west-1.compute.internal"
Oct 30 21:52:20.352112 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:20.352097    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-202.us-west-1.compute.internal"
Oct 30 21:52:20.687434 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:20.687396    2578 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Oct 30 21:52:20.687965 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:20.687580    2578 reflector.go:578] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:161: Unexpected watch close - watch lasted less than a second and no items received"
Oct 30 21:52:20.687965 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:20.687577    2578 reflector.go:578] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:161: Unexpected watch close - watch lasted less than a second and no items received"
Oct 30 21:52:20.687965 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:20.687593    2578 reflector.go:578] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.Service" err="very short watch: k8s.io/client-go/informers/factory.go:161: Unexpected watch close - watch lasted less than a second and no items received"
Oct 30 21:52:20.687965 ip-10-0-131-202 kubenswrapper[2578]: E1030 21:52:20.687646    2578 event.go:368] "Unable to write event (may retry after sleeping)" err="Patch \"https://a9adc942bda134c9d9d9f5a2e264780f-d8cf5ed739e19940.elb.us-east-2.amazonaws.com:6443/api/v1/namespaces/default/events/ip-10-0-131-202.us-west-1.compute.internal.18e36d3261cb6449\": read tcp 10.0.131.202:40236->77.112.174.59:6443: use of closed network connection" event="&Event{ObjectMeta:{ip-10-0-131-202.us-west-1.compute.internal.18e36d3261cb6449  default   4926 0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-131-202.us-west-1.compute.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-131-202.us-west-1.compute.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-131-202.us-west-1.compute.internal,},FirstTimestamp:2026-10-30 21:52:19 +0000 UTC,LastTimestamp:2026-10-30 21:52:19.935037829 +0000 UTC m=+0.574738042,Count:3,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-131-202.us-west-1.compute.internal,}"
Oct 30 21:52:20.735379 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:20.735360    2578 apiserver.go:52] "Watching apiserver"
Oct 30 21:52:20.760098 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:20.760070    2578 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Oct 30 21:52:20.969409 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:20.969374    2578 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-10-29 21:47:20 +0000 UTC" deadline="2028-07-18 11:27:00.159896494 +0000 UTC"
Oct 30 21:52:20.969409 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:20.969406    2578 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15037h34m39.190494303s"
Oct 30 21:52:20.992178 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:20.992151    2578 reflector.go:446] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Oct 30 21:52:20.992494 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:20.992472    2578 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-image-registry/node-ca-f9t6t","openshift-multus/network-metrics-daemon-wpdr9","kube-system/global-pull-secret-syncer-4vtdc","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nd9qk","openshift-network-diagnostics/network-check-target-slfvl","openshift-network-operator/iptables-alerter-2hzq7","openshift-ovn-kubernetes/ovnkube-node-lhj9s","kube-system/konnectivity-agent-x7l5l","kube-system/kube-apiserver-proxy-ip-10-0-131-202.us-west-1.compute.internal","openshift-cluster-node-tuning-operator/tuned-9zvbw","openshift-dns/node-resolver-7v7fb","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-202.us-west-1.compute.internal","openshift-multus/multus-additional-cni-plugins-vpbvr","openshift-multus/multus-hf5t4"]
Oct 30 21:52:20.994904 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:20.994888    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-f9t6t"
Oct 30 21:52:20.996955 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:20.996937    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wpdr9"
Oct 30 21:52:20.997049 ip-10-0-131-202 kubenswrapper[2578]: E1030 21:52:20.997022    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wpdr9" podUID="8806f1e1-0057-4738-8fbc-7e503e1f91ca"
Oct 30 21:52:20.998963 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:20.998947    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4vtdc"
Oct 30 21:52:20.999046 ip-10-0-131-202 kubenswrapper[2578]: E1030 21:52:20.999027    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4vtdc" podUID="d17b7754-d1d4-4952-97c7-7a04c714b48c"
Oct 30 21:52:21.000941 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.000929    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nd9qk"
Oct 30 21:52:21.002937 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.002923    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-slfvl"
Oct 30 21:52:21.002990 ip-10-0-131-202 kubenswrapper[2578]: E1030 21:52:21.002980    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-slfvl" podUID="7be00c31-281f-4d1c-8e12-1bf74bbe7cbd"
Oct 30 21:52:21.004927 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.004911    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-2hzq7"
Oct 30 21:52:21.007146 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.007134    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-lhj9s"
Oct 30 21:52:21.009938 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.009911    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-x7l5l"
Oct 30 21:52:21.012674 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.012655    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-9zvbw"
Oct 30 21:52:21.014992 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.014974    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-7v7fb"
Oct 30 21:52:21.017335 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.017320    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-vpbvr"
Oct 30 21:52:21.019572 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.019553    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-hf5t4"
Oct 30 21:52:21.047337 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.047313    2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-ww8ls\""
Oct 30 21:52:21.047426 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.047386    2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Oct 30 21:52:21.047467 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.047436    2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Oct 30 21:52:21.047556 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.047543    2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Oct 30 21:52:21.063545 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.063529    2578 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Oct 30 21:52:21.070108 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.070091    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/982efb96-3613-4ef2-ac51-ed3f40745a32-etc-sysconfig\") pod \"tuned-9zvbw\" (UID: \"982efb96-3613-4ef2-ac51-ed3f40745a32\") " pod="openshift-cluster-node-tuning-operator/tuned-9zvbw"
Oct 30 21:52:21.070205 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.070157    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/982efb96-3613-4ef2-ac51-ed3f40745a32-etc-sysctl-d\") pod \"tuned-9zvbw\" (UID: \"982efb96-3613-4ef2-ac51-ed3f40745a32\") " pod="openshift-cluster-node-tuning-operator/tuned-9zvbw"
Oct 30 21:52:21.070255 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.070203    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/982efb96-3613-4ef2-ac51-ed3f40745a32-run\") pod \"tuned-9zvbw\" (UID: \"982efb96-3613-4ef2-ac51-ed3f40745a32\") " pod="openshift-cluster-node-tuning-operator/tuned-9zvbw"
Oct 30 21:52:21.070295 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.070250    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/982efb96-3613-4ef2-ac51-ed3f40745a32-tmp\") pod \"tuned-9zvbw\" (UID: \"982efb96-3613-4ef2-ac51-ed3f40745a32\") " pod="openshift-cluster-node-tuning-operator/tuned-9zvbw"
Oct 30 21:52:21.070295 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.070288    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-78t6j\" (UniqueName: \"kubernetes.io/projected/164bafd8-ff17-46a2-afc0-dd01fa1aaeb6-kube-api-access-78t6j\") pod \"node-ca-f9t6t\" (UID: \"164bafd8-ff17-46a2-afc0-dd01fa1aaeb6\") " pod="openshift-image-registry/node-ca-f9t6t"
Oct 30 21:52:21.070375 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.070308    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/bf0e36d0-87bd-4e1e-be40-8a3a0f856987-run-ovn\") pod \"ovnkube-node-lhj9s\" (UID: \"bf0e36d0-87bd-4e1e-be40-8a3a0f856987\") " pod="openshift-ovn-kubernetes/ovnkube-node-lhj9s"
Oct 30 21:52:21.070375 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.070363    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/c590d5fe-57de-4905-90fe-8527b215ce94-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-vpbvr\" (UID: \"c590d5fe-57de-4905-90fe-8527b215ce94\") " pod="openshift-multus/multus-additional-cni-plugins-vpbvr"
Oct 30 21:52:21.070464 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.070398    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b1a02001-9121-47bd-ae96-b39196ad2357-cnibin\") pod \"multus-hf5t4\" (UID: \"b1a02001-9121-47bd-ae96-b39196ad2357\") " pod="openshift-multus/multus-hf5t4"
Oct 30 21:52:21.070464 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.070421    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bf0e36d0-87bd-4e1e-be40-8a3a0f856987-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-lhj9s\" (UID: \"bf0e36d0-87bd-4e1e-be40-8a3a0f856987\") " pod="openshift-ovn-kubernetes/ovnkube-node-lhj9s"
Oct 30 21:52:21.070548 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.070469    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/a4ff4cce-671b-4136-8b63-b22148c0db4e-iptables-alerter-script\") pod \"iptables-alerter-2hzq7\" (UID: \"a4ff4cce-671b-4136-8b63-b22148c0db4e\") " pod="openshift-network-operator/iptables-alerter-2hzq7"
Oct 30 21:52:21.070548 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.070500    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/bf0e36d0-87bd-4e1e-be40-8a3a0f856987-host-kubelet\") pod \"ovnkube-node-lhj9s\" (UID: \"bf0e36d0-87bd-4e1e-be40-8a3a0f856987\") " pod="openshift-ovn-kubernetes/ovnkube-node-lhj9s"
Oct 30 21:52:21.070548 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.070514    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bf0e36d0-87bd-4e1e-be40-8a3a0f856987-run-openvswitch\") pod \"ovnkube-node-lhj9s\" (UID: \"bf0e36d0-87bd-4e1e-be40-8a3a0f856987\") " pod="openshift-ovn-kubernetes/ovnkube-node-lhj9s"
Oct 30 21:52:21.070548 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.070544    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bf0e36d0-87bd-4e1e-be40-8a3a0f856987-host-run-ovn-kubernetes\") pod \"ovnkube-node-lhj9s\" (UID: \"bf0e36d0-87bd-4e1e-be40-8a3a0f856987\") " pod="openshift-ovn-kubernetes/ovnkube-node-lhj9s"
Oct 30 21:52:21.070704 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.070562    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/c590d5fe-57de-4905-90fe-8527b215ce94-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-vpbvr\" (UID: \"c590d5fe-57de-4905-90fe-8527b215ce94\") " pod="openshift-multus/multus-additional-cni-plugins-vpbvr"
Oct 30 21:52:21.070704 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.070586    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/b1a02001-9121-47bd-ae96-b39196ad2357-host-var-lib-cni-multus\") pod \"multus-hf5t4\" (UID: \"b1a02001-9121-47bd-ae96-b39196ad2357\") " pod="openshift-multus/multus-hf5t4"
Oct 30 21:52:21.070704 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.070622    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8806f1e1-0057-4738-8fbc-7e503e1f91ca-metrics-certs\") pod \"network-metrics-daemon-wpdr9\" (UID: \"8806f1e1-0057-4738-8fbc-7e503e1f91ca\") " pod="openshift-multus/network-metrics-daemon-wpdr9"
Oct 30 21:52:21.070704 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.070659    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/386f9569-a5f5-4a9e-beee-3005994186cd-socket-dir\") pod \"aws-ebs-csi-driver-node-nd9qk\" (UID: \"386f9569-a5f5-4a9e-beee-3005994186cd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nd9qk"
Oct 30 21:52:21.070704 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.070692    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/386f9569-a5f5-4a9e-beee-3005994186cd-registration-dir\") pod \"aws-ebs-csi-driver-node-nd9qk\" (UID: \"386f9569-a5f5-4a9e-beee-3005994186cd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nd9qk"
Oct 30 21:52:21.070930 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.070708    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/982efb96-3613-4ef2-ac51-ed3f40745a32-etc-modprobe-d\") pod \"tuned-9zvbw\" (UID: \"982efb96-3613-4ef2-ac51-ed3f40745a32\") " pod="openshift-cluster-node-tuning-operator/tuned-9zvbw"
Oct 30 21:52:21.070930 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.070732    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/dffd7a70-b7f3-4e1b-b3ab-74c5a574cab2-hosts-file\") pod \"node-resolver-7v7fb\" (UID: \"dffd7a70-b7f3-4e1b-b3ab-74c5a574cab2\") " pod="openshift-dns/node-resolver-7v7fb"
Oct 30 21:52:21.070930 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.070746    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dffd7a70-b7f3-4e1b-b3ab-74c5a574cab2-tmp-dir\") pod \"node-resolver-7v7fb\" (UID: \"dffd7a70-b7f3-4e1b-b3ab-74c5a574cab2\") " pod="openshift-dns/node-resolver-7v7fb"
Oct 30 21:52:21.070930 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.070762    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/164bafd8-ff17-46a2-afc0-dd01fa1aaeb6-serviceca\") pod \"node-ca-f9t6t\" (UID: \"164bafd8-ff17-46a2-afc0-dd01fa1aaeb6\") " pod="openshift-image-registry/node-ca-f9t6t"
Oct 30 21:52:21.070930 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.070784    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/b1a02001-9121-47bd-ae96-b39196ad2357-hostroot\") pod \"multus-hf5t4\" (UID: \"b1a02001-9121-47bd-ae96-b39196ad2357\") " pod="openshift-multus/multus-hf5t4"
Oct 30 21:52:21.070930 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.070800    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fwsxm\" (UniqueName: \"kubernetes.io/projected/8806f1e1-0057-4738-8fbc-7e503e1f91ca-kube-api-access-fwsxm\") pod \"network-metrics-daemon-wpdr9\" (UID: \"8806f1e1-0057-4738-8fbc-7e503e1f91ca\") " pod="openshift-multus/network-metrics-daemon-wpdr9"
Oct 30 21:52:21.070930 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.070832    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/bf0e36d0-87bd-4e1e-be40-8a3a0f856987-log-socket\") pod \"ovnkube-node-lhj9s\" (UID: \"bf0e36d0-87bd-4e1e-be40-8a3a0f856987\") " pod="openshift-ovn-kubernetes/ovnkube-node-lhj9s"
Oct 30 21:52:21.070930 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.070869    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/25590e1f-99e3-4469-8a69-39cd30317c28-agent-certs\") pod \"konnectivity-agent-x7l5l\" (UID: \"25590e1f-99e3-4469-8a69-39cd30317c28\") " pod="kube-system/konnectivity-agent-x7l5l"
Oct 30 21:52:21.070930 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.070906    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/982efb96-3613-4ef2-ac51-ed3f40745a32-host\") pod \"tuned-9zvbw\" (UID: \"982efb96-3613-4ef2-ac51-ed3f40745a32\") " pod="openshift-cluster-node-tuning-operator/tuned-9zvbw"
Oct 30 21:52:21.071325 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.070934    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/164bafd8-ff17-46a2-afc0-dd01fa1aaeb6-host\") pod \"node-ca-f9t6t\" (UID: \"164bafd8-ff17-46a2-afc0-dd01fa1aaeb6\") " pod="openshift-image-registry/node-ca-f9t6t"
Oct 30 21:52:21.071325 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.070950    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/b1a02001-9121-47bd-ae96-b39196ad2357-host-run-k8s-cni-cncf-io\") pod \"multus-hf5t4\" (UID: \"b1a02001-9121-47bd-ae96-b39196ad2357\") " pod="openshift-multus/multus-hf5t4"
Oct 30 21:52:21.071325 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.070968    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/386f9569-a5f5-4a9e-beee-3005994186cd-sys-fs\") pod \"aws-ebs-csi-driver-node-nd9qk\" (UID: \"386f9569-a5f5-4a9e-beee-3005994186cd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nd9qk"
Oct 30 21:52:21.071325 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.070982    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c590d5fe-57de-4905-90fe-8527b215ce94-tuning-conf-dir\") pod \"multus-additional-cni-plugins-vpbvr\" (UID: \"c590d5fe-57de-4905-90fe-8527b215ce94\") " pod="openshift-multus/multus-additional-cni-plugins-vpbvr"
Oct 30 21:52:21.071325 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.071004    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/982efb96-3613-4ef2-ac51-ed3f40745a32-etc-systemd\") pod \"tuned-9zvbw\" (UID: \"982efb96-3613-4ef2-ac51-ed3f40745a32\") " pod="openshift-cluster-node-tuning-operator/tuned-9zvbw"
Oct 30 21:52:21.071325 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.071021    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/bf0e36d0-87bd-4e1e-be40-8a3a0f856987-host-cni-netd\") pod \"ovnkube-node-lhj9s\" (UID: \"bf0e36d0-87bd-4e1e-be40-8a3a0f856987\") " pod="openshift-ovn-kubernetes/ovnkube-node-lhj9s"
Oct 30 21:52:21.071325 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.071039    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/386f9569-a5f5-4a9e-beee-3005994186cd-etc-selinux\") pod \"aws-ebs-csi-driver-node-nd9qk\" (UID: \"386f9569-a5f5-4a9e-beee-3005994186cd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nd9qk"
Oct 30 21:52:21.071325 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.071053    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6r8d9\" (UniqueName: \"kubernetes.io/projected/a4ff4cce-671b-4136-8b63-b22148c0db4e-kube-api-access-6r8d9\") pod \"iptables-alerter-2hzq7\" (UID: \"a4ff4cce-671b-4136-8b63-b22148c0db4e\") " pod="openshift-network-operator/iptables-alerter-2hzq7"
Oct 30 21:52:21.071325 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.071080    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/982efb96-3613-4ef2-ac51-ed3f40745a32-lib-modules\") pod \"tuned-9zvbw\" (UID: \"982efb96-3613-4ef2-ac51-ed3f40745a32\") " pod="openshift-cluster-node-tuning-operator/tuned-9zvbw"
Oct 30 21:52:21.071325 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.071113    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/982efb96-3613-4ef2-ac51-ed3f40745a32-var-lib-kubelet\") pod \"tuned-9zvbw\" (UID: \"982efb96-3613-4ef2-ac51-ed3f40745a32\") " pod="openshift-cluster-node-tuning-operator/tuned-9zvbw"
Oct 30 21:52:21.071325 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.071155    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/bf0e36d0-87bd-4e1e-be40-8a3a0f856987-systemd-units\") pod \"ovnkube-node-lhj9s\" (UID: \"bf0e36d0-87bd-4e1e-be40-8a3a0f856987\") " pod="openshift-ovn-kubernetes/ovnkube-node-lhj9s"
Oct 30 21:52:21.071325 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.071178    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/bf0e36d0-87bd-4e1e-be40-8a3a0f856987-host-run-netns\") pod \"ovnkube-node-lhj9s\" (UID: \"bf0e36d0-87bd-4e1e-be40-8a3a0f856987\") " pod="openshift-ovn-kubernetes/ovnkube-node-lhj9s"
Oct 30 21:52:21.071325 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.071201    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c590d5fe-57de-4905-90fe-8527b215ce94-system-cni-dir\") pod \"multus-additional-cni-plugins-vpbvr\" (UID: \"c590d5fe-57de-4905-90fe-8527b215ce94\") " pod="openshift-multus/multus-additional-cni-plugins-vpbvr"
Oct 30 21:52:21.071325 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.071227    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fq868\" (UniqueName: \"kubernetes.io/projected/c590d5fe-57de-4905-90fe-8527b215ce94-kube-api-access-fq868\") pod \"multus-additional-cni-plugins-vpbvr\" (UID: \"c590d5fe-57de-4905-90fe-8527b215ce94\") " pod="openshift-multus/multus-additional-cni-plugins-vpbvr"
Oct 30 21:52:21.071325 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.071254    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/bf0e36d0-87bd-4e1e-be40-8a3a0f856987-ovn-node-metrics-cert\") pod \"ovnkube-node-lhj9s\" (UID: \"bf0e36d0-87bd-4e1e-be40-8a3a0f856987\") " pod="openshift-ovn-kubernetes/ovnkube-node-lhj9s"
Oct 30 21:52:21.071325 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.071277    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ctg2b\" (UniqueName: \"kubernetes.io/projected/bf0e36d0-87bd-4e1e-be40-8a3a0f856987-kube-api-access-ctg2b\") pod \"ovnkube-node-lhj9s\" (UID: \"bf0e36d0-87bd-4e1e-be40-8a3a0f856987\") " pod="openshift-ovn-kubernetes/ovnkube-node-lhj9s"
Oct 30 21:52:21.071931 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.071305    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b1a02001-9121-47bd-ae96-b39196ad2357-multus-cni-dir\") pod \"multus-hf5t4\" (UID: \"b1a02001-9121-47bd-ae96-b39196ad2357\") " pod="openshift-multus/multus-hf5t4"
Oct 30 21:52:21.071931 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.071333    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b1a02001-9121-47bd-ae96-b39196ad2357-host-run-netns\") pod \"multus-hf5t4\" (UID: \"b1a02001-9121-47bd-ae96-b39196ad2357\") " pod="openshift-multus/multus-hf5t4"
Oct 30 21:52:21.071931 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.071362    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vsg28\" (UniqueName: \"kubernetes.io/projected/b1a02001-9121-47bd-ae96-b39196ad2357-kube-api-access-vsg28\") pod \"multus-hf5t4\" (UID: \"b1a02001-9121-47bd-ae96-b39196ad2357\") " pod="openshift-multus/multus-hf5t4"
Oct 30 21:52:21.071931 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.071392    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/bf0e36d0-87bd-4e1e-be40-8a3a0f856987-host-cni-bin\") pod \"ovnkube-node-lhj9s\" (UID: \"bf0e36d0-87bd-4e1e-be40-8a3a0f856987\") " pod="openshift-ovn-kubernetes/ovnkube-node-lhj9s"
Oct 30 21:52:21.071931 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.071426    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/d17b7754-d1d4-4952-97c7-7a04c714b48c-dbus\") pod \"global-pull-secret-syncer-4vtdc\" (UID: \"d17b7754-d1d4-4952-97c7-7a04c714b48c\") " pod="kube-system/global-pull-secret-syncer-4vtdc"
Oct 30 21:52:21.071931 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.071449    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/25590e1f-99e3-4469-8a69-39cd30317c28-konnectivity-ca\") pod \"konnectivity-agent-x7l5l\" (UID: \"25590e1f-99e3-4469-8a69-39cd30317c28\") " pod="kube-system/konnectivity-agent-x7l5l"
Oct 30 21:52:21.071931 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.071478    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/982efb96-3613-4ef2-ac51-ed3f40745a32-sys\") pod \"tuned-9zvbw\" (UID: \"982efb96-3613-4ef2-ac51-ed3f40745a32\") " pod="openshift-cluster-node-tuning-operator/tuned-9zvbw"
Oct 30 21:52:21.071931 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.071511    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/982efb96-3613-4ef2-ac51-ed3f40745a32-etc-tuned\") pod \"tuned-9zvbw\" (UID: \"982efb96-3613-4ef2-ac51-ed3f40745a32\") " pod="openshift-cluster-node-tuning-operator/tuned-9zvbw"
Oct 30 21:52:21.071931 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.071548    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bf0e36d0-87bd-4e1e-be40-8a3a0f856987-etc-openvswitch\") pod \"ovnkube-node-lhj9s\" (UID: \"bf0e36d0-87bd-4e1e-be40-8a3a0f856987\") " pod="openshift-ovn-kubernetes/ovnkube-node-lhj9s"
Oct 30 21:52:21.071931 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.071564    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b1a02001-9121-47bd-ae96-b39196ad2357-host-var-lib-kubelet\") pod \"multus-hf5t4\" (UID: \"b1a02001-9121-47bd-ae96-b39196ad2357\") " pod="openshift-multus/multus-hf5t4"
Oct 30 21:52:21.071931 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.071577    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/b1a02001-9121-47bd-ae96-b39196ad2357-multus-daemon-config\") pod \"multus-hf5t4\" (UID: \"b1a02001-9121-47bd-ae96-b39196ad2357\") " pod="openshift-multus/multus-hf5t4"
Oct 30 21:52:21.071931 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.071591    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/bf0e36d0-87bd-4e1e-be40-8a3a0f856987-env-overrides\") pod \"ovnkube-node-lhj9s\" (UID: \"bf0e36d0-87bd-4e1e-be40-8a3a0f856987\") " pod="openshift-ovn-kubernetes/ovnkube-node-lhj9s"
Oct 30 21:52:21.071931 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.071605    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/982efb96-3613-4ef2-ac51-ed3f40745a32-etc-sysctl-conf\") pod \"tuned-9zvbw\" (UID: \"982efb96-3613-4ef2-ac51-ed3f40745a32\") " pod="openshift-cluster-node-tuning-operator/tuned-9zvbw"
Oct 30 21:52:21.071931 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.071618    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/bf0e36d0-87bd-4e1e-be40-8a3a0f856987-ovnkube-script-lib\") pod \"ovnkube-node-lhj9s\" (UID: \"bf0e36d0-87bd-4e1e-be40-8a3a0f856987\") " pod="openshift-ovn-kubernetes/ovnkube-node-lhj9s"
Oct 30 21:52:21.071931 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.071633    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c590d5fe-57de-4905-90fe-8527b215ce94-cni-binary-copy\") pod \"multus-additional-cni-plugins-vpbvr\" (UID: \"c590d5fe-57de-4905-90fe-8527b215ce94\") " pod="openshift-multus/multus-additional-cni-plugins-vpbvr"
Oct 30 21:52:21.071931 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.071655    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/d17b7754-d1d4-4952-97c7-7a04c714b48c-kubelet-config\") pod \"global-pull-secret-syncer-4vtdc\" (UID: \"d17b7754-d1d4-4952-97c7-7a04c714b48c\") " pod="kube-system/global-pull-secret-syncer-4vtdc"
Oct 30 21:52:21.071931 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.071671    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hrlj5\" (UniqueName: \"kubernetes.io/projected/dffd7a70-b7f3-4e1b-b3ab-74c5a574cab2-kube-api-access-hrlj5\") pod \"node-resolver-7v7fb\" (UID: \"dffd7a70-b7f3-4e1b-b3ab-74c5a574cab2\") " pod="openshift-dns/node-resolver-7v7fb"
Oct 30 21:52:21.072511 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.071685    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/b1a02001-9121-47bd-ae96-b39196ad2357-host-run-multus-certs\") pod \"multus-hf5t4\" (UID: \"b1a02001-9121-47bd-ae96-b39196ad2357\") " pod="openshift-multus/multus-hf5t4"
Oct 30 21:52:21.072511 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.071707    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b1a02001-9121-47bd-ae96-b39196ad2357-etc-kubernetes\") pod \"multus-hf5t4\" (UID: \"b1a02001-9121-47bd-ae96-b39196ad2357\") " pod="openshift-multus/multus-hf5t4"
Oct 30 21:52:21.072511 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.071726    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d17b7754-d1d4-4952-97c7-7a04c714b48c-original-pull-secret\") pod \"global-pull-secret-syncer-4vtdc\" (UID: \"d17b7754-d1d4-4952-97c7-7a04c714b48c\") " pod="kube-system/global-pull-secret-syncer-4vtdc"
Oct 30 21:52:21.072511 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.071742    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/386f9569-a5f5-4a9e-beee-3005994186cd-kubelet-dir\") pod \"aws-ebs-csi-driver-node-nd9qk\" (UID: \"386f9569-a5f5-4a9e-beee-3005994186cd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nd9qk"
Oct 30 21:52:21.072511 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.071767    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/386f9569-a5f5-4a9e-beee-3005994186cd-device-dir\") pod \"aws-ebs-csi-driver-node-nd9qk\" (UID: \"386f9569-a5f5-4a9e-beee-3005994186cd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nd9qk"
Oct 30 21:52:21.072511 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.071795    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/982efb96-3613-4ef2-ac51-ed3f40745a32-etc-kubernetes\") pod \"tuned-9zvbw\" (UID: \"982efb96-3613-4ef2-ac51-ed3f40745a32\") " pod="openshift-cluster-node-tuning-operator/tuned-9zvbw"
Oct 30 21:52:21.072511 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.071827    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2pwzh\" (UniqueName: \"kubernetes.io/projected/982efb96-3613-4ef2-ac51-ed3f40745a32-kube-api-access-2pwzh\") pod \"tuned-9zvbw\" (UID: \"982efb96-3613-4ef2-ac51-ed3f40745a32\") " pod="openshift-cluster-node-tuning-operator/tuned-9zvbw"
Oct 30 21:52:21.072511 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.071854    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/bf0e36d0-87bd-4e1e-be40-8a3a0f856987-host-slash\") pod \"ovnkube-node-lhj9s\" (UID: \"bf0e36d0-87bd-4e1e-be40-8a3a0f856987\") " pod="openshift-ovn-kubernetes/ovnkube-node-lhj9s"
Oct 30 21:52:21.072511 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.071892    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bf0e36d0-87bd-4e1e-be40-8a3a0f856987-var-lib-openvswitch\") pod \"ovnkube-node-lhj9s\" (UID: \"bf0e36d0-87bd-4e1e-be40-8a3a0f856987\") " pod="openshift-ovn-kubernetes/ovnkube-node-lhj9s"
Oct 30 21:52:21.072511 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.071911    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b1a02001-9121-47bd-ae96-b39196ad2357-multus-conf-dir\") pod \"multus-hf5t4\" (UID: \"b1a02001-9121-47bd-ae96-b39196ad2357\") " pod="openshift-multus/multus-hf5t4"
Oct 30 21:52:21.072511 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.071928    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tg4f9\" (UniqueName: \"kubernetes.io/projected/386f9569-a5f5-4a9e-beee-3005994186cd-kube-api-access-tg4f9\") pod \"aws-ebs-csi-driver-node-nd9qk\" (UID: \"386f9569-a5f5-4a9e-beee-3005994186cd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nd9qk"
Oct 30 21:52:21.072511 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.071944    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a4ff4cce-671b-4136-8b63-b22148c0db4e-host-slash\") pod \"iptables-alerter-2hzq7\" (UID: \"a4ff4cce-671b-4136-8b63-b22148c0db4e\") " pod="openshift-network-operator/iptables-alerter-2hzq7"
Oct 30 21:52:21.072511 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.071961    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/bf0e36d0-87bd-4e1e-be40-8a3a0f856987-run-systemd\") pod \"ovnkube-node-lhj9s\" (UID: \"bf0e36d0-87bd-4e1e-be40-8a3a0f856987\") " pod="openshift-ovn-kubernetes/ovnkube-node-lhj9s"
Oct 30 21:52:21.072511 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.071975    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/bf0e36d0-87bd-4e1e-be40-8a3a0f856987-node-log\") pod \"ovnkube-node-lhj9s\" (UID: \"bf0e36d0-87bd-4e1e-be40-8a3a0f856987\") " pod="openshift-ovn-kubernetes/ovnkube-node-lhj9s"
Oct 30 21:52:21.072511 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.071993    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b1a02001-9121-47bd-ae96-b39196ad2357-cni-binary-copy\") pod \"multus-hf5t4\" (UID: \"b1a02001-9121-47bd-ae96-b39196ad2357\") " pod="openshift-multus/multus-hf5t4"
Oct 30 21:52:21.072511 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.072009    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6j99v\" (UniqueName: \"kubernetes.io/projected/7be00c31-281f-4d1c-8e12-1bf74bbe7cbd-kube-api-access-6j99v\") pod \"network-check-target-slfvl\" (UID: \"7be00c31-281f-4d1c-8e12-1bf74bbe7cbd\") " pod="openshift-network-diagnostics/network-check-target-slfvl"
Oct 30 21:52:21.072921 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.072035    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/bf0e36d0-87bd-4e1e-be40-8a3a0f856987-ovnkube-config\") pod \"ovnkube-node-lhj9s\" (UID: \"bf0e36d0-87bd-4e1e-be40-8a3a0f856987\") " pod="openshift-ovn-kubernetes/ovnkube-node-lhj9s"
Oct 30 21:52:21.072921 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.072067    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b1a02001-9121-47bd-ae96-b39196ad2357-system-cni-dir\") pod \"multus-hf5t4\" (UID: \"b1a02001-9121-47bd-ae96-b39196ad2357\") " pod="openshift-multus/multus-hf5t4"
Oct 30 21:52:21.072921 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.072098    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/b1a02001-9121-47bd-ae96-b39196ad2357-multus-socket-dir-parent\") pod \"multus-hf5t4\" (UID: \"b1a02001-9121-47bd-ae96-b39196ad2357\") " pod="openshift-multus/multus-hf5t4"
Oct 30 21:52:21.072921 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.072139    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b1a02001-9121-47bd-ae96-b39196ad2357-host-var-lib-cni-bin\") pod \"multus-hf5t4\" (UID: \"b1a02001-9121-47bd-ae96-b39196ad2357\") " pod="openshift-multus/multus-hf5t4"
Oct 30 21:52:21.072921 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.072162    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c590d5fe-57de-4905-90fe-8527b215ce94-cnibin\") pod \"multus-additional-cni-plugins-vpbvr\" (UID: \"c590d5fe-57de-4905-90fe-8527b215ce94\") " pod="openshift-multus/multus-additional-cni-plugins-vpbvr"
Oct 30 21:52:21.144706 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.144682    2578 reflector.go:446] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Oct 30 21:52:21.144800 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.144738    2578 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-l97s4"
Oct 30 21:52:21.147362 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.147344    2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Oct 30 21:52:21.147710 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.147689    2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Oct 30 21:52:21.147710 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.147701    2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Oct 30 21:52:21.147851 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.147749    2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent-dockercfg-ppl5h\""
Oct 30 21:52:21.147851 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.147749    2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-node-identity-dockercfg-gqgtq\""
Oct 30 21:52:21.147851 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.147756    2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-vk8r7\""
Oct 30 21:52:21.147851 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.147795    2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-q4htg\""
Oct 30 21:52:21.147851 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.147817    2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-gg6dx\""
Oct 30 21:52:21.148035 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.147859    2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-bdqch\""
Oct 30 21:52:21.148035 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.147868    2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-hqgxz\""
Oct 30 21:52:21.148035 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.147940    2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-mgstz\""
Oct 30 21:52:21.148509 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.148490    2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Oct 30 21:52:21.148602 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.148504    2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Oct 30 21:52:21.148602 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.148562    2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Oct 30 21:52:21.148713 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.148634    2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Oct 30 21:52:21.148713 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.148626    2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Oct 30 21:52:21.148713 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.148656    2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Oct 30 21:52:21.148713 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.148682    2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Oct 30 21:52:21.148713 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.148712    2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Oct 30 21:52:21.148910 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.148754    2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Oct 30 21:52:21.148910 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.148818    2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Oct 30 21:52:21.148910 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.148841    2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Oct 30 21:52:21.149003 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.148993    2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Oct 30 21:52:21.149251 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.149240    2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Oct 30 21:52:21.156285 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.156267    2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Oct 30 21:52:21.156752 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.156639    2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Oct 30 21:52:21.156752 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.156713    2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Oct 30 21:52:21.156752 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.156715    2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Oct 30 21:52:21.172894 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.172874    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/bf0e36d0-87bd-4e1e-be40-8a3a0f856987-host-run-netns\") pod \"ovnkube-node-lhj9s\" (UID: \"bf0e36d0-87bd-4e1e-be40-8a3a0f856987\") " pod="openshift-ovn-kubernetes/ovnkube-node-lhj9s"
Oct 30 21:52:21.172986 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.172903    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c590d5fe-57de-4905-90fe-8527b215ce94-system-cni-dir\") pod \"multus-additional-cni-plugins-vpbvr\" (UID: \"c590d5fe-57de-4905-90fe-8527b215ce94\") " pod="openshift-multus/multus-additional-cni-plugins-vpbvr"
Oct 30 21:52:21.172986 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.172920    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fq868\" (UniqueName: \"kubernetes.io/projected/c590d5fe-57de-4905-90fe-8527b215ce94-kube-api-access-fq868\") pod \"multus-additional-cni-plugins-vpbvr\" (UID: \"c590d5fe-57de-4905-90fe-8527b215ce94\") " pod="openshift-multus/multus-additional-cni-plugins-vpbvr"
Oct 30 21:52:21.172986 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.172935    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/bf0e36d0-87bd-4e1e-be40-8a3a0f856987-ovn-node-metrics-cert\") pod \"ovnkube-node-lhj9s\" (UID: \"bf0e36d0-87bd-4e1e-be40-8a3a0f856987\") " pod="openshift-ovn-kubernetes/ovnkube-node-lhj9s"
Oct 30 21:52:21.173136 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.172993    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c590d5fe-57de-4905-90fe-8527b215ce94-system-cni-dir\") pod \"multus-additional-cni-plugins-vpbvr\" (UID: \"c590d5fe-57de-4905-90fe-8527b215ce94\") " pod="openshift-multus/multus-additional-cni-plugins-vpbvr"
Oct 30 21:52:21.173136 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.173004    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/bf0e36d0-87bd-4e1e-be40-8a3a0f856987-host-run-netns\") pod \"ovnkube-node-lhj9s\" (UID: \"bf0e36d0-87bd-4e1e-be40-8a3a0f856987\") " pod="openshift-ovn-kubernetes/ovnkube-node-lhj9s"
Oct 30 21:52:21.173136 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.173047    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ctg2b\" (UniqueName: \"kubernetes.io/projected/bf0e36d0-87bd-4e1e-be40-8a3a0f856987-kube-api-access-ctg2b\") pod \"ovnkube-node-lhj9s\" (UID: \"bf0e36d0-87bd-4e1e-be40-8a3a0f856987\") " pod="openshift-ovn-kubernetes/ovnkube-node-lhj9s"
Oct 30 21:52:21.173136 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.173080    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b1a02001-9121-47bd-ae96-b39196ad2357-multus-cni-dir\") pod \"multus-hf5t4\" (UID: \"b1a02001-9121-47bd-ae96-b39196ad2357\") " pod="openshift-multus/multus-hf5t4"
Oct 30 21:52:21.173136 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.173102    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b1a02001-9121-47bd-ae96-b39196ad2357-host-run-netns\") pod \"multus-hf5t4\" (UID: \"b1a02001-9121-47bd-ae96-b39196ad2357\") " pod="openshift-multus/multus-hf5t4"
Oct 30 21:52:21.173363 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.173140    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vsg28\" (UniqueName: \"kubernetes.io/projected/b1a02001-9121-47bd-ae96-b39196ad2357-kube-api-access-vsg28\") pod \"multus-hf5t4\" (UID: \"b1a02001-9121-47bd-ae96-b39196ad2357\") " pod="openshift-multus/multus-hf5t4"
Oct 30 21:52:21.173363 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.173164    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/bf0e36d0-87bd-4e1e-be40-8a3a0f856987-host-cni-bin\") pod \"ovnkube-node-lhj9s\" (UID: \"bf0e36d0-87bd-4e1e-be40-8a3a0f856987\") " pod="openshift-ovn-kubernetes/ovnkube-node-lhj9s"
Oct 30 21:52:21.173363 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.173181    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b1a02001-9121-47bd-ae96-b39196ad2357-host-run-netns\") pod \"multus-hf5t4\" (UID: \"b1a02001-9121-47bd-ae96-b39196ad2357\") " pod="openshift-multus/multus-hf5t4"
Oct 30 21:52:21.173363 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.173187    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/d17b7754-d1d4-4952-97c7-7a04c714b48c-dbus\") pod \"global-pull-secret-syncer-4vtdc\" (UID: \"d17b7754-d1d4-4952-97c7-7a04c714b48c\") " pod="kube-system/global-pull-secret-syncer-4vtdc"
Oct 30 21:52:21.173363 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.173222    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/bf0e36d0-87bd-4e1e-be40-8a3a0f856987-host-cni-bin\") pod \"ovnkube-node-lhj9s\" (UID: \"bf0e36d0-87bd-4e1e-be40-8a3a0f856987\") " pod="openshift-ovn-kubernetes/ovnkube-node-lhj9s"
Oct 30 21:52:21.173363 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.173240    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/25590e1f-99e3-4469-8a69-39cd30317c28-konnectivity-ca\") pod \"konnectivity-agent-x7l5l\" (UID: \"25590e1f-99e3-4469-8a69-39cd30317c28\") " pod="kube-system/konnectivity-agent-x7l5l"
Oct 30 21:52:21.173363 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.173219    2578 swap_util.go:78] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Oct 30 21:52:21.173363 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.173268    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/982efb96-3613-4ef2-ac51-ed3f40745a32-sys\") pod \"tuned-9zvbw\" (UID: \"982efb96-3613-4ef2-ac51-ed3f40745a32\") " pod="openshift-cluster-node-tuning-operator/tuned-9zvbw"
Oct 30 21:52:21.173363 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.173283    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/d17b7754-d1d4-4952-97c7-7a04c714b48c-dbus\") pod \"global-pull-secret-syncer-4vtdc\" (UID: \"d17b7754-d1d4-4952-97c7-7a04c714b48c\") " pod="kube-system/global-pull-secret-syncer-4vtdc"
Oct 30 21:52:21.173363 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.173281    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b1a02001-9121-47bd-ae96-b39196ad2357-multus-cni-dir\") pod \"multus-hf5t4\" (UID: \"b1a02001-9121-47bd-ae96-b39196ad2357\") " pod="openshift-multus/multus-hf5t4"
Oct 30 21:52:21.173363 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.173338    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/982efb96-3613-4ef2-ac51-ed3f40745a32-sys\") pod \"tuned-9zvbw\" (UID: \"982efb96-3613-4ef2-ac51-ed3f40745a32\") " pod="openshift-cluster-node-tuning-operator/tuned-9zvbw"
Oct 30 21:52:21.173852 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.173375    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/982efb96-3613-4ef2-ac51-ed3f40745a32-etc-tuned\") pod \"tuned-9zvbw\" (UID: \"982efb96-3613-4ef2-ac51-ed3f40745a32\") " pod="openshift-cluster-node-tuning-operator/tuned-9zvbw"
Oct 30 21:52:21.173852 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.173408    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bf0e36d0-87bd-4e1e-be40-8a3a0f856987-etc-openvswitch\") pod \"ovnkube-node-lhj9s\" (UID: \"bf0e36d0-87bd-4e1e-be40-8a3a0f856987\") " pod="openshift-ovn-kubernetes/ovnkube-node-lhj9s"
Oct 30 21:52:21.173852 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.173436    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b1a02001-9121-47bd-ae96-b39196ad2357-host-var-lib-kubelet\") pod \"multus-hf5t4\" (UID: \"b1a02001-9121-47bd-ae96-b39196ad2357\") " pod="openshift-multus/multus-hf5t4"
Oct 30 21:52:21.173852 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.173445    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bf0e36d0-87bd-4e1e-be40-8a3a0f856987-etc-openvswitch\") pod \"ovnkube-node-lhj9s\" (UID: \"bf0e36d0-87bd-4e1e-be40-8a3a0f856987\") " pod="openshift-ovn-kubernetes/ovnkube-node-lhj9s"
Oct 30 21:52:21.173852 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.173459    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/b1a02001-9121-47bd-ae96-b39196ad2357-multus-daemon-config\") pod \"multus-hf5t4\" (UID: \"b1a02001-9121-47bd-ae96-b39196ad2357\") " pod="openshift-multus/multus-hf5t4"
Oct 30 21:52:21.173852 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.173482    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/bf0e36d0-87bd-4e1e-be40-8a3a0f856987-env-overrides\") pod \"ovnkube-node-lhj9s\" (UID: \"bf0e36d0-87bd-4e1e-be40-8a3a0f856987\") " pod="openshift-ovn-kubernetes/ovnkube-node-lhj9s"
Oct 30 21:52:21.173852 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.173497    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b1a02001-9121-47bd-ae96-b39196ad2357-host-var-lib-kubelet\") pod \"multus-hf5t4\" (UID: \"b1a02001-9121-47bd-ae96-b39196ad2357\") " pod="openshift-multus/multus-hf5t4"
Oct 30 21:52:21.173852 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.173505    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/982efb96-3613-4ef2-ac51-ed3f40745a32-etc-sysctl-conf\") pod \"tuned-9zvbw\" (UID: \"982efb96-3613-4ef2-ac51-ed3f40745a32\") " pod="openshift-cluster-node-tuning-operator/tuned-9zvbw"
Oct 30 21:52:21.173852 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.173526    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/bf0e36d0-87bd-4e1e-be40-8a3a0f856987-ovnkube-script-lib\") pod \"ovnkube-node-lhj9s\" (UID: \"bf0e36d0-87bd-4e1e-be40-8a3a0f856987\") " pod="openshift-ovn-kubernetes/ovnkube-node-lhj9s"
Oct 30 21:52:21.173852 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.173541    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c590d5fe-57de-4905-90fe-8527b215ce94-cni-binary-copy\") pod \"multus-additional-cni-plugins-vpbvr\" (UID: \"c590d5fe-57de-4905-90fe-8527b215ce94\") " pod="openshift-multus/multus-additional-cni-plugins-vpbvr"
Oct 30 21:52:21.173852 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.173556    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/d17b7754-d1d4-4952-97c7-7a04c714b48c-kubelet-config\") pod \"global-pull-secret-syncer-4vtdc\" (UID: \"d17b7754-d1d4-4952-97c7-7a04c714b48c\") " pod="kube-system/global-pull-secret-syncer-4vtdc"
Oct 30 21:52:21.173852 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.173573    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hrlj5\" (UniqueName: \"kubernetes.io/projected/dffd7a70-b7f3-4e1b-b3ab-74c5a574cab2-kube-api-access-hrlj5\") pod \"node-resolver-7v7fb\" (UID: \"dffd7a70-b7f3-4e1b-b3ab-74c5a574cab2\") " pod="openshift-dns/node-resolver-7v7fb"
Oct 30 21:52:21.173852 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.173601    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/b1a02001-9121-47bd-ae96-b39196ad2357-host-run-multus-certs\") pod \"multus-hf5t4\" (UID: \"b1a02001-9121-47bd-ae96-b39196ad2357\") " pod="openshift-multus/multus-hf5t4"
Oct 30 21:52:21.173852 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.173624    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b1a02001-9121-47bd-ae96-b39196ad2357-etc-kubernetes\") pod \"multus-hf5t4\" (UID: \"b1a02001-9121-47bd-ae96-b39196ad2357\") " pod="openshift-multus/multus-hf5t4"
Oct 30 21:52:21.173852 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.173647    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d17b7754-d1d4-4952-97c7-7a04c714b48c-original-pull-secret\") pod \"global-pull-secret-syncer-4vtdc\" (UID: \"d17b7754-d1d4-4952-97c7-7a04c714b48c\") " pod="kube-system/global-pull-secret-syncer-4vtdc"
Oct 30 21:52:21.173852 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.173669    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/386f9569-a5f5-4a9e-beee-3005994186cd-kubelet-dir\") pod \"aws-ebs-csi-driver-node-nd9qk\" (UID: \"386f9569-a5f5-4a9e-beee-3005994186cd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nd9qk"
Oct 30 21:52:21.173852 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.173692    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/386f9569-a5f5-4a9e-beee-3005994186cd-device-dir\") pod \"aws-ebs-csi-driver-node-nd9qk\" (UID: \"386f9569-a5f5-4a9e-beee-3005994186cd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nd9qk"
Oct 30 21:52:21.174780 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.173713    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/982efb96-3613-4ef2-ac51-ed3f40745a32-etc-kubernetes\") pod \"tuned-9zvbw\" (UID: \"982efb96-3613-4ef2-ac51-ed3f40745a32\") " pod="openshift-cluster-node-tuning-operator/tuned-9zvbw"
Oct 30 21:52:21.174780 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.173738    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2pwzh\" (UniqueName: \"kubernetes.io/projected/982efb96-3613-4ef2-ac51-ed3f40745a32-kube-api-access-2pwzh\") pod \"tuned-9zvbw\" (UID: \"982efb96-3613-4ef2-ac51-ed3f40745a32\") " pod="openshift-cluster-node-tuning-operator/tuned-9zvbw"
Oct 30 21:52:21.174780 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.173742    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/982efb96-3613-4ef2-ac51-ed3f40745a32-etc-sysctl-conf\") pod \"tuned-9zvbw\" (UID: \"982efb96-3613-4ef2-ac51-ed3f40745a32\") " pod="openshift-cluster-node-tuning-operator/tuned-9zvbw"
Oct 30 21:52:21.174780 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.173761    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/bf0e36d0-87bd-4e1e-be40-8a3a0f856987-host-slash\") pod \"ovnkube-node-lhj9s\" (UID: \"bf0e36d0-87bd-4e1e-be40-8a3a0f856987\") " pod="openshift-ovn-kubernetes/ovnkube-node-lhj9s"
Oct 30 21:52:21.174780 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.173776    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/25590e1f-99e3-4469-8a69-39cd30317c28-konnectivity-ca\") pod \"konnectivity-agent-x7l5l\" (UID: \"25590e1f-99e3-4469-8a69-39cd30317c28\") " pod="kube-system/konnectivity-agent-x7l5l"
Oct 30 21:52:21.174780 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.173785    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bf0e36d0-87bd-4e1e-be40-8a3a0f856987-var-lib-openvswitch\") pod \"ovnkube-node-lhj9s\" (UID: \"bf0e36d0-87bd-4e1e-be40-8a3a0f856987\") " pod="openshift-ovn-kubernetes/ovnkube-node-lhj9s"
Oct 30 21:52:21.174780 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.173807    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b1a02001-9121-47bd-ae96-b39196ad2357-multus-conf-dir\") pod \"multus-hf5t4\" (UID: \"b1a02001-9121-47bd-ae96-b39196ad2357\") " pod="openshift-multus/multus-hf5t4"
Oct 30 21:52:21.174780 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.173831    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tg4f9\" (UniqueName: \"kubernetes.io/projected/386f9569-a5f5-4a9e-beee-3005994186cd-kube-api-access-tg4f9\") pod \"aws-ebs-csi-driver-node-nd9qk\" (UID: \"386f9569-a5f5-4a9e-beee-3005994186cd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nd9qk"
Oct 30 21:52:21.174780 ip-10-0-131-202 kubenswrapper[2578]: E1030 21:52:21.173835    2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 30 21:52:21.174780 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.173851    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/982efb96-3613-4ef2-ac51-ed3f40745a32-etc-kubernetes\") pod \"tuned-9zvbw\" (UID: \"982efb96-3613-4ef2-ac51-ed3f40745a32\") " pod="openshift-cluster-node-tuning-operator/tuned-9zvbw"
Oct 30 21:52:21.174780 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.173854    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a4ff4cce-671b-4136-8b63-b22148c0db4e-host-slash\") pod \"iptables-alerter-2hzq7\" (UID: \"a4ff4cce-671b-4136-8b63-b22148c0db4e\") " pod="openshift-network-operator/iptables-alerter-2hzq7"
Oct 30 21:52:21.174780 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.173889    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/bf0e36d0-87bd-4e1e-be40-8a3a0f856987-run-systemd\") pod \"ovnkube-node-lhj9s\" (UID: \"bf0e36d0-87bd-4e1e-be40-8a3a0f856987\") " pod="openshift-ovn-kubernetes/ovnkube-node-lhj9s"
Oct 30 21:52:21.174780 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.173894    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a4ff4cce-671b-4136-8b63-b22148c0db4e-host-slash\") pod \"iptables-alerter-2hzq7\" (UID: \"a4ff4cce-671b-4136-8b63-b22148c0db4e\") " pod="openshift-network-operator/iptables-alerter-2hzq7"
Oct 30 21:52:21.174780 ip-10-0-131-202 kubenswrapper[2578]: E1030 21:52:21.173909    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d17b7754-d1d4-4952-97c7-7a04c714b48c-original-pull-secret podName:d17b7754-d1d4-4952-97c7-7a04c714b48c nodeName:}" failed. No retries permitted until 2026-10-30 21:52:21.673888451 +0000 UTC m=+2.313588669 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d17b7754-d1d4-4952-97c7-7a04c714b48c-original-pull-secret") pod "global-pull-secret-syncer-4vtdc" (UID: "d17b7754-d1d4-4952-97c7-7a04c714b48c") : object "kube-system"/"original-pull-secret" not registered
Oct 30 21:52:21.174780 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.173918    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/bf0e36d0-87bd-4e1e-be40-8a3a0f856987-run-systemd\") pod \"ovnkube-node-lhj9s\" (UID: \"bf0e36d0-87bd-4e1e-be40-8a3a0f856987\") " pod="openshift-ovn-kubernetes/ovnkube-node-lhj9s"
Oct 30 21:52:21.174780 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.173943    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/bf0e36d0-87bd-4e1e-be40-8a3a0f856987-node-log\") pod \"ovnkube-node-lhj9s\" (UID: \"bf0e36d0-87bd-4e1e-be40-8a3a0f856987\") " pod="openshift-ovn-kubernetes/ovnkube-node-lhj9s"
Oct 30 21:52:21.174780 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.173961    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/386f9569-a5f5-4a9e-beee-3005994186cd-kubelet-dir\") pod \"aws-ebs-csi-driver-node-nd9qk\" (UID: \"386f9569-a5f5-4a9e-beee-3005994186cd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nd9qk"
Oct 30 21:52:21.175642 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.173969    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b1a02001-9121-47bd-ae96-b39196ad2357-cni-binary-copy\") pod \"multus-hf5t4\" (UID: \"b1a02001-9121-47bd-ae96-b39196ad2357\") " pod="openshift-multus/multus-hf5t4"
Oct 30 21:52:21.175642 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.174000    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/386f9569-a5f5-4a9e-beee-3005994186cd-device-dir\") pod \"aws-ebs-csi-driver-node-nd9qk\" (UID: \"386f9569-a5f5-4a9e-beee-3005994186cd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nd9qk"
Oct 30 21:52:21.175642 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.174049    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/d17b7754-d1d4-4952-97c7-7a04c714b48c-kubelet-config\") pod \"global-pull-secret-syncer-4vtdc\" (UID: \"d17b7754-d1d4-4952-97c7-7a04c714b48c\") " pod="kube-system/global-pull-secret-syncer-4vtdc"
Oct 30 21:52:21.175642 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.174076    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/bf0e36d0-87bd-4e1e-be40-8a3a0f856987-env-overrides\") pod \"ovnkube-node-lhj9s\" (UID: \"bf0e36d0-87bd-4e1e-be40-8a3a0f856987\") " pod="openshift-ovn-kubernetes/ovnkube-node-lhj9s"
Oct 30 21:52:21.175642 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.174095    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c590d5fe-57de-4905-90fe-8527b215ce94-cni-binary-copy\") pod \"multus-additional-cni-plugins-vpbvr\" (UID: \"c590d5fe-57de-4905-90fe-8527b215ce94\") " pod="openshift-multus/multus-additional-cni-plugins-vpbvr"
Oct 30 21:52:21.175642 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.174151    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/bf0e36d0-87bd-4e1e-be40-8a3a0f856987-host-slash\") pod \"ovnkube-node-lhj9s\" (UID: \"bf0e36d0-87bd-4e1e-be40-8a3a0f856987\") " pod="openshift-ovn-kubernetes/ovnkube-node-lhj9s"
Oct 30 21:52:21.175642 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.174158    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b1a02001-9121-47bd-ae96-b39196ad2357-multus-conf-dir\") pod \"multus-hf5t4\" (UID: \"b1a02001-9121-47bd-ae96-b39196ad2357\") " pod="openshift-multus/multus-hf5t4"
Oct 30 21:52:21.175642 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.173998    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6j99v\" (UniqueName: \"kubernetes.io/projected/7be00c31-281f-4d1c-8e12-1bf74bbe7cbd-kube-api-access-6j99v\") pod \"network-check-target-slfvl\" (UID: \"7be00c31-281f-4d1c-8e12-1bf74bbe7cbd\") " pod="openshift-network-diagnostics/network-check-target-slfvl"
Oct 30 21:52:21.175642 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.174192    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b1a02001-9121-47bd-ae96-b39196ad2357-etc-kubernetes\") pod \"multus-hf5t4\" (UID: \"b1a02001-9121-47bd-ae96-b39196ad2357\") " pod="openshift-multus/multus-hf5t4"
Oct 30 21:52:21.175642 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.174209    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/bf0e36d0-87bd-4e1e-be40-8a3a0f856987-ovnkube-config\") pod \"ovnkube-node-lhj9s\" (UID: \"bf0e36d0-87bd-4e1e-be40-8a3a0f856987\") " pod="openshift-ovn-kubernetes/ovnkube-node-lhj9s"
Oct 30 21:52:21.175642 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.174230    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b1a02001-9121-47bd-ae96-b39196ad2357-system-cni-dir\") pod \"multus-hf5t4\" (UID: \"b1a02001-9121-47bd-ae96-b39196ad2357\") " pod="openshift-multus/multus-hf5t4"
Oct 30 21:52:21.175642 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.174251    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/b1a02001-9121-47bd-ae96-b39196ad2357-multus-socket-dir-parent\") pod \"multus-hf5t4\" (UID: \"b1a02001-9121-47bd-ae96-b39196ad2357\") " pod="openshift-multus/multus-hf5t4"
Oct 30 21:52:21.175642 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.174261    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bf0e36d0-87bd-4e1e-be40-8a3a0f856987-var-lib-openvswitch\") pod \"ovnkube-node-lhj9s\" (UID: \"bf0e36d0-87bd-4e1e-be40-8a3a0f856987\") " pod="openshift-ovn-kubernetes/ovnkube-node-lhj9s"
Oct 30 21:52:21.175642 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.174273    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b1a02001-9121-47bd-ae96-b39196ad2357-host-var-lib-cni-bin\") pod \"multus-hf5t4\" (UID: \"b1a02001-9121-47bd-ae96-b39196ad2357\") " pod="openshift-multus/multus-hf5t4"
Oct 30 21:52:21.175642 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.174318    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b1a02001-9121-47bd-ae96-b39196ad2357-host-var-lib-cni-bin\") pod \"multus-hf5t4\" (UID: \"b1a02001-9121-47bd-ae96-b39196ad2357\") " pod="openshift-multus/multus-hf5t4"
Oct 30 21:52:21.175642 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.174321    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/b1a02001-9121-47bd-ae96-b39196ad2357-multus-socket-dir-parent\") pod \"multus-hf5t4\" (UID: \"b1a02001-9121-47bd-ae96-b39196ad2357\") " pod="openshift-multus/multus-hf5t4"
Oct 30 21:52:21.175642 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.174231    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/bf0e36d0-87bd-4e1e-be40-8a3a0f856987-node-log\") pod \"ovnkube-node-lhj9s\" (UID: \"bf0e36d0-87bd-4e1e-be40-8a3a0f856987\") " pod="openshift-ovn-kubernetes/ovnkube-node-lhj9s"
Oct 30 21:52:21.175642 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.174349    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c590d5fe-57de-4905-90fe-8527b215ce94-cnibin\") pod \"multus-additional-cni-plugins-vpbvr\" (UID: \"c590d5fe-57de-4905-90fe-8527b215ce94\") " pod="openshift-multus/multus-additional-cni-plugins-vpbvr"
Oct 30 21:52:21.176454 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.174366    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b1a02001-9121-47bd-ae96-b39196ad2357-system-cni-dir\") pod \"multus-hf5t4\" (UID: \"b1a02001-9121-47bd-ae96-b39196ad2357\") " pod="openshift-multus/multus-hf5t4"
Oct 30 21:52:21.176454 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.174374    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/982efb96-3613-4ef2-ac51-ed3f40745a32-etc-sysconfig\") pod \"tuned-9zvbw\" (UID: \"982efb96-3613-4ef2-ac51-ed3f40745a32\") " pod="openshift-cluster-node-tuning-operator/tuned-9zvbw"
Oct 30 21:52:21.176454 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.174405    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/982efb96-3613-4ef2-ac51-ed3f40745a32-etc-sysctl-d\") pod \"tuned-9zvbw\" (UID: \"982efb96-3613-4ef2-ac51-ed3f40745a32\") " pod="openshift-cluster-node-tuning-operator/tuned-9zvbw"
Oct 30 21:52:21.176454 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.174409    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/982efb96-3613-4ef2-ac51-ed3f40745a32-etc-sysconfig\") pod \"tuned-9zvbw\" (UID: \"982efb96-3613-4ef2-ac51-ed3f40745a32\") " pod="openshift-cluster-node-tuning-operator/tuned-9zvbw"
Oct 30 21:52:21.176454 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.174426    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/982efb96-3613-4ef2-ac51-ed3f40745a32-run\") pod \"tuned-9zvbw\" (UID: \"982efb96-3613-4ef2-ac51-ed3f40745a32\") " pod="openshift-cluster-node-tuning-operator/tuned-9zvbw"
Oct 30 21:52:21.176454 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.174449    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c590d5fe-57de-4905-90fe-8527b215ce94-cnibin\") pod \"multus-additional-cni-plugins-vpbvr\" (UID: \"c590d5fe-57de-4905-90fe-8527b215ce94\") " pod="openshift-multus/multus-additional-cni-plugins-vpbvr"
Oct 30 21:52:21.176454 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.174455    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/982efb96-3613-4ef2-ac51-ed3f40745a32-tmp\") pod \"tuned-9zvbw\" (UID: \"982efb96-3613-4ef2-ac51-ed3f40745a32\") " pod="openshift-cluster-node-tuning-operator/tuned-9zvbw"
Oct 30 21:52:21.176454 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.174481    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-78t6j\" (UniqueName: \"kubernetes.io/projected/164bafd8-ff17-46a2-afc0-dd01fa1aaeb6-kube-api-access-78t6j\") pod \"node-ca-f9t6t\" (UID: \"164bafd8-ff17-46a2-afc0-dd01fa1aaeb6\") " pod="openshift-image-registry/node-ca-f9t6t"
Oct 30 21:52:21.176454 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.174506    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/bf0e36d0-87bd-4e1e-be40-8a3a0f856987-run-ovn\") pod \"ovnkube-node-lhj9s\" (UID: \"bf0e36d0-87bd-4e1e-be40-8a3a0f856987\") " pod="openshift-ovn-kubernetes/ovnkube-node-lhj9s"
Oct 30 21:52:21.176454 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.174531    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/c590d5fe-57de-4905-90fe-8527b215ce94-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-vpbvr\" (UID: \"c590d5fe-57de-4905-90fe-8527b215ce94\") " pod="openshift-multus/multus-additional-cni-plugins-vpbvr"
Oct 30 21:52:21.176454 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.174563    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/982efb96-3613-4ef2-ac51-ed3f40745a32-etc-sysctl-d\") pod \"tuned-9zvbw\" (UID: \"982efb96-3613-4ef2-ac51-ed3f40745a32\") " pod="openshift-cluster-node-tuning-operator/tuned-9zvbw"
Oct 30 21:52:21.176454 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.174555    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b1a02001-9121-47bd-ae96-b39196ad2357-cnibin\") pod \"multus-hf5t4\" (UID: \"b1a02001-9121-47bd-ae96-b39196ad2357\") " pod="openshift-multus/multus-hf5t4"
Oct 30 21:52:21.176454 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.174604    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bf0e36d0-87bd-4e1e-be40-8a3a0f856987-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-lhj9s\" (UID: \"bf0e36d0-87bd-4e1e-be40-8a3a0f856987\") " pod="openshift-ovn-kubernetes/ovnkube-node-lhj9s"
Oct 30 21:52:21.176454 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.174615    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/982efb96-3613-4ef2-ac51-ed3f40745a32-run\") pod \"tuned-9zvbw\" (UID: \"982efb96-3613-4ef2-ac51-ed3f40745a32\") " pod="openshift-cluster-node-tuning-operator/tuned-9zvbw"
Oct 30 21:52:21.176454 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.174634    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/a4ff4cce-671b-4136-8b63-b22148c0db4e-iptables-alerter-script\") pod \"iptables-alerter-2hzq7\" (UID: \"a4ff4cce-671b-4136-8b63-b22148c0db4e\") " pod="openshift-network-operator/iptables-alerter-2hzq7"
Oct 30 21:52:21.176454 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.174661    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/bf0e36d0-87bd-4e1e-be40-8a3a0f856987-host-kubelet\") pod \"ovnkube-node-lhj9s\" (UID: \"bf0e36d0-87bd-4e1e-be40-8a3a0f856987\") " pod="openshift-ovn-kubernetes/ovnkube-node-lhj9s"
Oct 30 21:52:21.176454 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.174682    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bf0e36d0-87bd-4e1e-be40-8a3a0f856987-run-openvswitch\") pod \"ovnkube-node-lhj9s\" (UID: \"bf0e36d0-87bd-4e1e-be40-8a3a0f856987\") " pod="openshift-ovn-kubernetes/ovnkube-node-lhj9s"
Oct 30 21:52:21.177110 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.174709    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bf0e36d0-87bd-4e1e-be40-8a3a0f856987-host-run-ovn-kubernetes\") pod \"ovnkube-node-lhj9s\" (UID: \"bf0e36d0-87bd-4e1e-be40-8a3a0f856987\") " pod="openshift-ovn-kubernetes/ovnkube-node-lhj9s"
Oct 30 21:52:21.177110 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.174198    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/b1a02001-9121-47bd-ae96-b39196ad2357-host-run-multus-certs\") pod \"multus-hf5t4\" (UID: \"b1a02001-9121-47bd-ae96-b39196ad2357\") " pod="openshift-multus/multus-hf5t4"
Oct 30 21:52:21.177110 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.174733    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/c590d5fe-57de-4905-90fe-8527b215ce94-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-vpbvr\" (UID: \"c590d5fe-57de-4905-90fe-8527b215ce94\") " pod="openshift-multus/multus-additional-cni-plugins-vpbvr"
Oct 30 21:52:21.177110 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.174757    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/b1a02001-9121-47bd-ae96-b39196ad2357-host-var-lib-cni-multus\") pod \"multus-hf5t4\" (UID: \"b1a02001-9121-47bd-ae96-b39196ad2357\") " pod="openshift-multus/multus-hf5t4"
Oct 30 21:52:21.177110 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.174775    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/bf0e36d0-87bd-4e1e-be40-8a3a0f856987-run-ovn\") pod \"ovnkube-node-lhj9s\" (UID: \"bf0e36d0-87bd-4e1e-be40-8a3a0f856987\") " pod="openshift-ovn-kubernetes/ovnkube-node-lhj9s"
Oct 30 21:52:21.177110 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.174781    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8806f1e1-0057-4738-8fbc-7e503e1f91ca-metrics-certs\") pod \"network-metrics-daemon-wpdr9\" (UID: \"8806f1e1-0057-4738-8fbc-7e503e1f91ca\") " pod="openshift-multus/network-metrics-daemon-wpdr9"
Oct 30 21:52:21.177110 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.174806    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/386f9569-a5f5-4a9e-beee-3005994186cd-socket-dir\") pod \"aws-ebs-csi-driver-node-nd9qk\" (UID: \"386f9569-a5f5-4a9e-beee-3005994186cd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nd9qk"
Oct 30 21:52:21.177110 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.174814    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/bf0e36d0-87bd-4e1e-be40-8a3a0f856987-host-kubelet\") pod \"ovnkube-node-lhj9s\" (UID: \"bf0e36d0-87bd-4e1e-be40-8a3a0f856987\") " pod="openshift-ovn-kubernetes/ovnkube-node-lhj9s"
Oct 30 21:52:21.177110 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.174829    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/386f9569-a5f5-4a9e-beee-3005994186cd-registration-dir\") pod \"aws-ebs-csi-driver-node-nd9qk\" (UID: \"386f9569-a5f5-4a9e-beee-3005994186cd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nd9qk"
Oct 30 21:52:21.177110 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.174852    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/982efb96-3613-4ef2-ac51-ed3f40745a32-etc-modprobe-d\") pod \"tuned-9zvbw\" (UID: \"982efb96-3613-4ef2-ac51-ed3f40745a32\") " pod="openshift-cluster-node-tuning-operator/tuned-9zvbw"
Oct 30 21:52:21.177110 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.174876    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/dffd7a70-b7f3-4e1b-b3ab-74c5a574cab2-hosts-file\") pod \"node-resolver-7v7fb\" (UID: \"dffd7a70-b7f3-4e1b-b3ab-74c5a574cab2\") " pod="openshift-dns/node-resolver-7v7fb"
Oct 30 21:52:21.177110 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.174901    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dffd7a70-b7f3-4e1b-b3ab-74c5a574cab2-tmp-dir\") pod \"node-resolver-7v7fb\" (UID: \"dffd7a70-b7f3-4e1b-b3ab-74c5a574cab2\") " pod="openshift-dns/node-resolver-7v7fb"
Oct 30 21:52:21.177110 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.174925    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/164bafd8-ff17-46a2-afc0-dd01fa1aaeb6-serviceca\") pod \"node-ca-f9t6t\" (UID: \"164bafd8-ff17-46a2-afc0-dd01fa1aaeb6\") " pod="openshift-image-registry/node-ca-f9t6t"
Oct 30 21:52:21.177110 ip-10-0-131-202 kubenswrapper[2578]: E1030 21:52:21.174942    2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 21:52:21.177110 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.174946    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/b1a02001-9121-47bd-ae96-b39196ad2357-hostroot\") pod \"multus-hf5t4\" (UID: \"b1a02001-9121-47bd-ae96-b39196ad2357\") " pod="openshift-multus/multus-hf5t4"
Oct 30 21:52:21.177110 ip-10-0-131-202 kubenswrapper[2578]: E1030 21:52:21.174993    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8806f1e1-0057-4738-8fbc-7e503e1f91ca-metrics-certs podName:8806f1e1-0057-4738-8fbc-7e503e1f91ca nodeName:}" failed. No retries permitted until 2026-10-30 21:52:21.674975901 +0000 UTC m=+2.314676117 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8806f1e1-0057-4738-8fbc-7e503e1f91ca-metrics-certs") pod "network-metrics-daemon-wpdr9" (UID: "8806f1e1-0057-4738-8fbc-7e503e1f91ca") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 21:52:21.177110 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.175012    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fwsxm\" (UniqueName: \"kubernetes.io/projected/8806f1e1-0057-4738-8fbc-7e503e1f91ca-kube-api-access-fwsxm\") pod \"network-metrics-daemon-wpdr9\" (UID: \"8806f1e1-0057-4738-8fbc-7e503e1f91ca\") " pod="openshift-multus/network-metrics-daemon-wpdr9"
Oct 30 21:52:21.177654 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.175020    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/b1a02001-9121-47bd-ae96-b39196ad2357-hostroot\") pod \"multus-hf5t4\" (UID: \"b1a02001-9121-47bd-ae96-b39196ad2357\") " pod="openshift-multus/multus-hf5t4"
Oct 30 21:52:21.177654 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.175038    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/bf0e36d0-87bd-4e1e-be40-8a3a0f856987-log-socket\") pod \"ovnkube-node-lhj9s\" (UID: \"bf0e36d0-87bd-4e1e-be40-8a3a0f856987\") " pod="openshift-ovn-kubernetes/ovnkube-node-lhj9s"
Oct 30 21:52:21.177654 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.175063    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/25590e1f-99e3-4469-8a69-39cd30317c28-agent-certs\") pod \"konnectivity-agent-x7l5l\" (UID: \"25590e1f-99e3-4469-8a69-39cd30317c28\") " pod="kube-system/konnectivity-agent-x7l5l"
Oct 30 21:52:21.177654 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.175088    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/982efb96-3613-4ef2-ac51-ed3f40745a32-host\") pod \"tuned-9zvbw\" (UID: \"982efb96-3613-4ef2-ac51-ed3f40745a32\") " pod="openshift-cluster-node-tuning-operator/tuned-9zvbw"
Oct 30 21:52:21.177654 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.175104    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/164bafd8-ff17-46a2-afc0-dd01fa1aaeb6-host\") pod \"node-ca-f9t6t\" (UID: \"164bafd8-ff17-46a2-afc0-dd01fa1aaeb6\") " pod="openshift-image-registry/node-ca-f9t6t"
Oct 30 21:52:21.177654 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.175145    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/b1a02001-9121-47bd-ae96-b39196ad2357-host-run-k8s-cni-cncf-io\") pod \"multus-hf5t4\" (UID: \"b1a02001-9121-47bd-ae96-b39196ad2357\") " pod="openshift-multus/multus-hf5t4"
Oct 30 21:52:21.177654 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.175168    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/386f9569-a5f5-4a9e-beee-3005994186cd-sys-fs\") pod \"aws-ebs-csi-driver-node-nd9qk\" (UID: \"386f9569-a5f5-4a9e-beee-3005994186cd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nd9qk"
Oct 30 21:52:21.177654 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.175193    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c590d5fe-57de-4905-90fe-8527b215ce94-tuning-conf-dir\") pod \"multus-additional-cni-plugins-vpbvr\" (UID: \"c590d5fe-57de-4905-90fe-8527b215ce94\") " pod="openshift-multus/multus-additional-cni-plugins-vpbvr"
Oct 30 21:52:21.177654 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.175195    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/386f9569-a5f5-4a9e-beee-3005994186cd-registration-dir\") pod \"aws-ebs-csi-driver-node-nd9qk\" (UID: \"386f9569-a5f5-4a9e-beee-3005994186cd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nd9qk"
Oct 30 21:52:21.177654 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.175206    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/bf0e36d0-87bd-4e1e-be40-8a3a0f856987-ovnkube-config\") pod \"ovnkube-node-lhj9s\" (UID: \"bf0e36d0-87bd-4e1e-be40-8a3a0f856987\") " pod="openshift-ovn-kubernetes/ovnkube-node-lhj9s"
Oct 30 21:52:21.177654 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.175219    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/982efb96-3613-4ef2-ac51-ed3f40745a32-etc-systemd\") pod \"tuned-9zvbw\" (UID: \"982efb96-3613-4ef2-ac51-ed3f40745a32\") " pod="openshift-cluster-node-tuning-operator/tuned-9zvbw"
Oct 30 21:52:21.177654 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.175235    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b1a02001-9121-47bd-ae96-b39196ad2357-cni-binary-copy\") pod \"multus-hf5t4\" (UID: \"b1a02001-9121-47bd-ae96-b39196ad2357\") " pod="openshift-multus/multus-hf5t4"
Oct 30 21:52:21.177654 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.175276    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/982efb96-3613-4ef2-ac51-ed3f40745a32-etc-modprobe-d\") pod \"tuned-9zvbw\" (UID: \"982efb96-3613-4ef2-ac51-ed3f40745a32\") " pod="openshift-cluster-node-tuning-operator/tuned-9zvbw"
Oct 30 21:52:21.177654 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.174851    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bf0e36d0-87bd-4e1e-be40-8a3a0f856987-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-lhj9s\" (UID: \"bf0e36d0-87bd-4e1e-be40-8a3a0f856987\") " pod="openshift-ovn-kubernetes/ovnkube-node-lhj9s"
Oct 30 21:52:21.177654 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.175295    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/bf0e36d0-87bd-4e1e-be40-8a3a0f856987-host-cni-netd\") pod \"ovnkube-node-lhj9s\" (UID: \"bf0e36d0-87bd-4e1e-be40-8a3a0f856987\") " pod="openshift-ovn-kubernetes/ovnkube-node-lhj9s"
Oct 30 21:52:21.177654 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.175295    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bf0e36d0-87bd-4e1e-be40-8a3a0f856987-run-openvswitch\") pod \"ovnkube-node-lhj9s\" (UID: \"bf0e36d0-87bd-4e1e-be40-8a3a0f856987\") " pod="openshift-ovn-kubernetes/ovnkube-node-lhj9s"
Oct 30 21:52:21.177654 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.175241    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/bf0e36d0-87bd-4e1e-be40-8a3a0f856987-host-cni-netd\") pod \"ovnkube-node-lhj9s\" (UID: \"bf0e36d0-87bd-4e1e-be40-8a3a0f856987\") " pod="openshift-ovn-kubernetes/ovnkube-node-lhj9s"
Oct 30 21:52:21.178173 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.175325    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bf0e36d0-87bd-4e1e-be40-8a3a0f856987-host-run-ovn-kubernetes\") pod \"ovnkube-node-lhj9s\" (UID: \"bf0e36d0-87bd-4e1e-be40-8a3a0f856987\") " pod="openshift-ovn-kubernetes/ovnkube-node-lhj9s"
Oct 30 21:52:21.178173 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.175347    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/386f9569-a5f5-4a9e-beee-3005994186cd-etc-selinux\") pod \"aws-ebs-csi-driver-node-nd9qk\" (UID: \"386f9569-a5f5-4a9e-beee-3005994186cd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nd9qk"
Oct 30 21:52:21.178173 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.175367    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/982efb96-3613-4ef2-ac51-ed3f40745a32-host\") pod \"tuned-9zvbw\" (UID: \"982efb96-3613-4ef2-ac51-ed3f40745a32\") " pod="openshift-cluster-node-tuning-operator/tuned-9zvbw"
Oct 30 21:52:21.178173 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.175375    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6r8d9\" (UniqueName: \"kubernetes.io/projected/a4ff4cce-671b-4136-8b63-b22148c0db4e-kube-api-access-6r8d9\") pod \"iptables-alerter-2hzq7\" (UID: \"a4ff4cce-671b-4136-8b63-b22148c0db4e\") " pod="openshift-network-operator/iptables-alerter-2hzq7"
Oct 30 21:52:21.178173 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.175399    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/982efb96-3613-4ef2-ac51-ed3f40745a32-lib-modules\") pod \"tuned-9zvbw\" (UID: \"982efb96-3613-4ef2-ac51-ed3f40745a32\") " pod="openshift-cluster-node-tuning-operator/tuned-9zvbw"
Oct 30 21:52:21.178173 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.174610    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b1a02001-9121-47bd-ae96-b39196ad2357-cnibin\") pod \"multus-hf5t4\" (UID: \"b1a02001-9121-47bd-ae96-b39196ad2357\") " pod="openshift-multus/multus-hf5t4"
Oct 30 21:52:21.178173 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.175423    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/982efb96-3613-4ef2-ac51-ed3f40745a32-var-lib-kubelet\") pod \"tuned-9zvbw\" (UID: \"982efb96-3613-4ef2-ac51-ed3f40745a32\") " pod="openshift-cluster-node-tuning-operator/tuned-9zvbw"
Oct 30 21:52:21.178173 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.175431    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/b1a02001-9121-47bd-ae96-b39196ad2357-host-run-k8s-cni-cncf-io\") pod \"multus-hf5t4\" (UID: \"b1a02001-9121-47bd-ae96-b39196ad2357\") " pod="openshift-multus/multus-hf5t4"
Oct 30 21:52:21.178173 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.175446    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/bf0e36d0-87bd-4e1e-be40-8a3a0f856987-systemd-units\") pod \"ovnkube-node-lhj9s\" (UID: \"bf0e36d0-87bd-4e1e-be40-8a3a0f856987\") " pod="openshift-ovn-kubernetes/ovnkube-node-lhj9s"
Oct 30 21:52:21.178173 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.175481    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/386f9569-a5f5-4a9e-beee-3005994186cd-sys-fs\") pod \"aws-ebs-csi-driver-node-nd9qk\" (UID: \"386f9569-a5f5-4a9e-beee-3005994186cd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nd9qk"
Oct 30 21:52:21.178173 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.175525    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/bf0e36d0-87bd-4e1e-be40-8a3a0f856987-systemd-units\") pod \"ovnkube-node-lhj9s\" (UID: \"bf0e36d0-87bd-4e1e-be40-8a3a0f856987\") " pod="openshift-ovn-kubernetes/ovnkube-node-lhj9s"
Oct 30 21:52:21.178173 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.175600    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c590d5fe-57de-4905-90fe-8527b215ce94-tuning-conf-dir\") pod \"multus-additional-cni-plugins-vpbvr\" (UID: \"c590d5fe-57de-4905-90fe-8527b215ce94\") " pod="openshift-multus/multus-additional-cni-plugins-vpbvr"
Oct 30 21:52:21.178173 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.175618    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/982efb96-3613-4ef2-ac51-ed3f40745a32-lib-modules\") pod \"tuned-9zvbw\" (UID: \"982efb96-3613-4ef2-ac51-ed3f40745a32\") " pod="openshift-cluster-node-tuning-operator/tuned-9zvbw"
Oct 30 21:52:21.178173 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.175400    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/164bafd8-ff17-46a2-afc0-dd01fa1aaeb6-host\") pod \"node-ca-f9t6t\" (UID: \"164bafd8-ff17-46a2-afc0-dd01fa1aaeb6\") " pod="openshift-image-registry/node-ca-f9t6t"
Oct 30 21:52:21.178173 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.175649    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/982efb96-3613-4ef2-ac51-ed3f40745a32-etc-systemd\") pod \"tuned-9zvbw\" (UID: \"982efb96-3613-4ef2-ac51-ed3f40745a32\") " pod="openshift-cluster-node-tuning-operator/tuned-9zvbw"
Oct 30 21:52:21.178173 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.175672    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/982efb96-3613-4ef2-ac51-ed3f40745a32-var-lib-kubelet\") pod \"tuned-9zvbw\" (UID: \"982efb96-3613-4ef2-ac51-ed3f40745a32\") " pod="openshift-cluster-node-tuning-operator/tuned-9zvbw"
Oct 30 21:52:21.178173 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.175680    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dffd7a70-b7f3-4e1b-b3ab-74c5a574cab2-tmp-dir\") pod \"node-resolver-7v7fb\" (UID: \"dffd7a70-b7f3-4e1b-b3ab-74c5a574cab2\") " pod="openshift-dns/node-resolver-7v7fb"
Oct 30 21:52:21.178173 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.175721    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/dffd7a70-b7f3-4e1b-b3ab-74c5a574cab2-hosts-file\") pod \"node-resolver-7v7fb\" (UID: \"dffd7a70-b7f3-4e1b-b3ab-74c5a574cab2\") " pod="openshift-dns/node-resolver-7v7fb"
Oct 30 21:52:21.178642 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.175821    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/386f9569-a5f5-4a9e-beee-3005994186cd-socket-dir\") pod \"aws-ebs-csi-driver-node-nd9qk\" (UID: \"386f9569-a5f5-4a9e-beee-3005994186cd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nd9qk"
Oct 30 21:52:21.178642 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.175856    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/386f9569-a5f5-4a9e-beee-3005994186cd-etc-selinux\") pod \"aws-ebs-csi-driver-node-nd9qk\" (UID: \"386f9569-a5f5-4a9e-beee-3005994186cd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nd9qk"
Oct 30 21:52:21.178642 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.176105    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/b1a02001-9121-47bd-ae96-b39196ad2357-host-var-lib-cni-multus\") pod \"multus-hf5t4\" (UID: \"b1a02001-9121-47bd-ae96-b39196ad2357\") " pod="openshift-multus/multus-hf5t4"
Oct 30 21:52:21.178642 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.176141    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/b1a02001-9121-47bd-ae96-b39196ad2357-multus-daemon-config\") pod \"multus-hf5t4\" (UID: \"b1a02001-9121-47bd-ae96-b39196ad2357\") " pod="openshift-multus/multus-hf5t4"
Oct 30 21:52:21.178642 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.176163    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/bf0e36d0-87bd-4e1e-be40-8a3a0f856987-log-socket\") pod \"ovnkube-node-lhj9s\" (UID: \"bf0e36d0-87bd-4e1e-be40-8a3a0f856987\") " pod="openshift-ovn-kubernetes/ovnkube-node-lhj9s"
Oct 30 21:52:21.178642 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.176248    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/c590d5fe-57de-4905-90fe-8527b215ce94-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-vpbvr\" (UID: \"c590d5fe-57de-4905-90fe-8527b215ce94\") " pod="openshift-multus/multus-additional-cni-plugins-vpbvr"
Oct 30 21:52:21.178642 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.176381    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/164bafd8-ff17-46a2-afc0-dd01fa1aaeb6-serviceca\") pod \"node-ca-f9t6t\" (UID: \"164bafd8-ff17-46a2-afc0-dd01fa1aaeb6\") " pod="openshift-image-registry/node-ca-f9t6t"
Oct 30 21:52:21.178642 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.176449    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/c590d5fe-57de-4905-90fe-8527b215ce94-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-vpbvr\" (UID: \"c590d5fe-57de-4905-90fe-8527b215ce94\") " pod="openshift-multus/multus-additional-cni-plugins-vpbvr"
Oct 30 21:52:21.178642 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.177160    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/982efb96-3613-4ef2-ac51-ed3f40745a32-etc-tuned\") pod \"tuned-9zvbw\" (UID: \"982efb96-3613-4ef2-ac51-ed3f40745a32\") " pod="openshift-cluster-node-tuning-operator/tuned-9zvbw"
Oct 30 21:52:21.178642 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.177359    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/982efb96-3613-4ef2-ac51-ed3f40745a32-tmp\") pod \"tuned-9zvbw\" (UID: \"982efb96-3613-4ef2-ac51-ed3f40745a32\") " pod="openshift-cluster-node-tuning-operator/tuned-9zvbw"
Oct 30 21:52:21.178642 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.177366    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/bf0e36d0-87bd-4e1e-be40-8a3a0f856987-ovn-node-metrics-cert\") pod \"ovnkube-node-lhj9s\" (UID: \"bf0e36d0-87bd-4e1e-be40-8a3a0f856987\") " pod="openshift-ovn-kubernetes/ovnkube-node-lhj9s"
Oct 30 21:52:21.178642 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.178347    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/25590e1f-99e3-4469-8a69-39cd30317c28-agent-certs\") pod \"konnectivity-agent-x7l5l\" (UID: \"25590e1f-99e3-4469-8a69-39cd30317c28\") " pod="kube-system/konnectivity-agent-x7l5l"
Oct 30 21:52:21.194278 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.194257    2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Oct 30 21:52:21.194369 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.194260    2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Oct 30 21:52:21.194369 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.194288    2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Oct 30 21:52:21.194369 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.194325    2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Oct 30 21:52:21.196109 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.196092    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/a4ff4cce-671b-4136-8b63-b22148c0db4e-iptables-alerter-script\") pod \"iptables-alerter-2hzq7\" (UID: \"a4ff4cce-671b-4136-8b63-b22148c0db4e\") " pod="openshift-network-operator/iptables-alerter-2hzq7"
Oct 30 21:52:21.204932 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.204908    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/bf0e36d0-87bd-4e1e-be40-8a3a0f856987-ovnkube-script-lib\") pod \"ovnkube-node-lhj9s\" (UID: \"bf0e36d0-87bd-4e1e-be40-8a3a0f856987\") " pod="openshift-ovn-kubernetes/ovnkube-node-lhj9s"
Oct 30 21:52:21.233392 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.233329    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fq868\" (UniqueName: \"kubernetes.io/projected/c590d5fe-57de-4905-90fe-8527b215ce94-kube-api-access-fq868\") pod \"multus-additional-cni-plugins-vpbvr\" (UID: \"c590d5fe-57de-4905-90fe-8527b215ce94\") " pod="openshift-multus/multus-additional-cni-plugins-vpbvr"
Oct 30 21:52:21.233491 ip-10-0-131-202 kubenswrapper[2578]: E1030 21:52:21.233396    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 30 21:52:21.233491 ip-10-0-131-202 kubenswrapper[2578]: E1030 21:52:21.233413    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 30 21:52:21.233491 ip-10-0-131-202 kubenswrapper[2578]: E1030 21:52:21.233423    2578 projected.go:196] Error preparing data for projected volume kube-api-access-6j99v for pod openshift-network-diagnostics/network-check-target-slfvl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 21:52:21.233491 ip-10-0-131-202 kubenswrapper[2578]: E1030 21:52:21.233474    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7be00c31-281f-4d1c-8e12-1bf74bbe7cbd-kube-api-access-6j99v podName:7be00c31-281f-4d1c-8e12-1bf74bbe7cbd nodeName:}" failed. No retries permitted until 2026-10-30 21:52:21.733461729 +0000 UTC m=+2.373161957 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-6j99v" (UniqueName: "kubernetes.io/projected/7be00c31-281f-4d1c-8e12-1bf74bbe7cbd-kube-api-access-6j99v") pod "network-check-target-slfvl" (UID: "7be00c31-281f-4d1c-8e12-1bf74bbe7cbd") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 21:52:21.233789 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.233768    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ctg2b\" (UniqueName: \"kubernetes.io/projected/bf0e36d0-87bd-4e1e-be40-8a3a0f856987-kube-api-access-ctg2b\") pod \"ovnkube-node-lhj9s\" (UID: \"bf0e36d0-87bd-4e1e-be40-8a3a0f856987\") " pod="openshift-ovn-kubernetes/ovnkube-node-lhj9s"
Oct 30 21:52:21.234547 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.234529    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2pwzh\" (UniqueName: \"kubernetes.io/projected/982efb96-3613-4ef2-ac51-ed3f40745a32-kube-api-access-2pwzh\") pod \"tuned-9zvbw\" (UID: \"982efb96-3613-4ef2-ac51-ed3f40745a32\") " pod="openshift-cluster-node-tuning-operator/tuned-9zvbw"
Oct 30 21:52:21.236132 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.236060    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6r8d9\" (UniqueName: \"kubernetes.io/projected/a4ff4cce-671b-4136-8b63-b22148c0db4e-kube-api-access-6r8d9\") pod \"iptables-alerter-2hzq7\" (UID: \"a4ff4cce-671b-4136-8b63-b22148c0db4e\") " pod="openshift-network-operator/iptables-alerter-2hzq7"
Oct 30 21:52:21.236478 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.236176    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vsg28\" (UniqueName: \"kubernetes.io/projected/b1a02001-9121-47bd-ae96-b39196ad2357-kube-api-access-vsg28\") pod \"multus-hf5t4\" (UID: \"b1a02001-9121-47bd-ae96-b39196ad2357\") " pod="openshift-multus/multus-hf5t4"
Oct 30 21:52:21.237896 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.237876    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hrlj5\" (UniqueName: \"kubernetes.io/projected/dffd7a70-b7f3-4e1b-b3ab-74c5a574cab2-kube-api-access-hrlj5\") pod \"node-resolver-7v7fb\" (UID: \"dffd7a70-b7f3-4e1b-b3ab-74c5a574cab2\") " pod="openshift-dns/node-resolver-7v7fb"
Oct 30 21:52:21.238033 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.238016    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-78t6j\" (UniqueName: \"kubernetes.io/projected/164bafd8-ff17-46a2-afc0-dd01fa1aaeb6-kube-api-access-78t6j\") pod \"node-ca-f9t6t\" (UID: \"164bafd8-ff17-46a2-afc0-dd01fa1aaeb6\") " pod="openshift-image-registry/node-ca-f9t6t"
Oct 30 21:52:21.238573 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.238555    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fwsxm\" (UniqueName: \"kubernetes.io/projected/8806f1e1-0057-4738-8fbc-7e503e1f91ca-kube-api-access-fwsxm\") pod \"network-metrics-daemon-wpdr9\" (UID: \"8806f1e1-0057-4738-8fbc-7e503e1f91ca\") " pod="openshift-multus/network-metrics-daemon-wpdr9"
Oct 30 21:52:21.238966 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.238949    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tg4f9\" (UniqueName: \"kubernetes.io/projected/386f9569-a5f5-4a9e-beee-3005994186cd-kube-api-access-tg4f9\") pod \"aws-ebs-csi-driver-node-nd9qk\" (UID: \"386f9569-a5f5-4a9e-beee-3005994186cd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nd9qk"
Oct 30 21:52:21.329786 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.329743    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-f9t6t"
Oct 30 21:52:21.339108 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.339080    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nd9qk"
Oct 30 21:52:21.358587 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.358559    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-2hzq7"
Oct 30 21:52:21.394719 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.394693    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-x7l5l"
Oct 30 21:52:21.394815 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.394759    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-lhj9s"
Oct 30 21:52:21.398926 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.398909    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-9zvbw"
Oct 30 21:52:21.413408 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.413389    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-7v7fb"
Oct 30 21:52:21.427584 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:21.427553    2578 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbf0e36d0_87bd_4e1e_be40_8a3a0f856987.slice/crio-437d7c0fb64e2862943d48eb6c7731f8d740ca3387b8624fad4e28e5b29e1511 WatchSource:0}: Error finding container 437d7c0fb64e2862943d48eb6c7731f8d740ca3387b8624fad4e28e5b29e1511: Status 404 returned error can't find the container with id 437d7c0fb64e2862943d48eb6c7731f8d740ca3387b8624fad4e28e5b29e1511
Oct 30 21:52:21.428322 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:21.428279    2578 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod982efb96_3613_4ef2_ac51_ed3f40745a32.slice/crio-6d0727caaaf3fa2870ab13c51cbab65fbfbb7687ef8ff9096a16e38f970f0243 WatchSource:0}: Error finding container 6d0727caaaf3fa2870ab13c51cbab65fbfbb7687ef8ff9096a16e38f970f0243: Status 404 returned error can't find the container with id 6d0727caaaf3fa2870ab13c51cbab65fbfbb7687ef8ff9096a16e38f970f0243
Oct 30 21:52:21.428899 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:21.428872    2578 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod04337a6f3666f0af07c3d2b702c6e379.slice/crio-693c46ffe0fe055558dabec72ff633a79b91a6e9ec57f3e27c1c31178984aaa7 WatchSource:0}: Error finding container 693c46ffe0fe055558dabec72ff633a79b91a6e9ec57f3e27c1c31178984aaa7: Status 404 returned error can't find the container with id 693c46ffe0fe055558dabec72ff633a79b91a6e9ec57f3e27c1c31178984aaa7
Oct 30 21:52:21.430027 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:21.430002    2578 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddffd7a70_b7f3_4e1b_b3ab_74c5a574cab2.slice/crio-da317ccc84703e328e24540d39e76dcf76ce091d7baad369756dba3796160cfb WatchSource:0}: Error finding container da317ccc84703e328e24540d39e76dcf76ce091d7baad369756dba3796160cfb: Status 404 returned error can't find the container with id da317ccc84703e328e24540d39e76dcf76ce091d7baad369756dba3796160cfb
Oct 30 21:52:21.430664 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:21.430645    2578 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podec1e59c1a0a508a7c75743ab25eff8ba.slice/crio-9751c27f53d33b0e6f09e8d175690fc83db3fe0e1f08314102e30aca663fe0c9 WatchSource:0}: Error finding container 9751c27f53d33b0e6f09e8d175690fc83db3fe0e1f08314102e30aca663fe0c9: Status 404 returned error can't find the container with id 9751c27f53d33b0e6f09e8d175690fc83db3fe0e1f08314102e30aca663fe0c9
Oct 30 21:52:21.431473 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:21.431324    2578 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod25590e1f_99e3_4469_8a69_39cd30317c28.slice/crio-a9887498d2723b7a9e696b07db4f758f0c7f19897e1b590ceb5739927bce13cb WatchSource:0}: Error finding container a9887498d2723b7a9e696b07db4f758f0c7f19897e1b590ceb5739927bce13cb: Status 404 returned error can't find the container with id a9887498d2723b7a9e696b07db4f758f0c7f19897e1b590ceb5739927bce13cb
Oct 30 21:52:21.433611 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.433583    2578 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 21:52:21.437421 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.437406    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-hf5t4"
Oct 30 21:52:21.442150 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.442136    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-vpbvr"
Oct 30 21:52:21.445922 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:21.445894    2578 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb1a02001_9121_47bd_ae96_b39196ad2357.slice/crio-3f2b3c7943e258b620f38697047406863aa88efda8d5faada68af4c4e955aeb5 WatchSource:0}: Error finding container 3f2b3c7943e258b620f38697047406863aa88efda8d5faada68af4c4e955aeb5: Status 404 returned error can't find the container with id 3f2b3c7943e258b620f38697047406863aa88efda8d5faada68af4c4e955aeb5
Oct 30 21:52:21.450313 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:21.450296    2578 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc590d5fe_57de_4905_90fe_8527b215ce94.slice/crio-6e596f1f222be642b03bca48eb0148c607deb4bab720ed68a09e44e31f2c0dea WatchSource:0}: Error finding container 6e596f1f222be642b03bca48eb0148c607deb4bab720ed68a09e44e31f2c0dea: Status 404 returned error can't find the container with id 6e596f1f222be642b03bca48eb0148c607deb4bab720ed68a09e44e31f2c0dea
Oct 30 21:52:21.678402 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.678370    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d17b7754-d1d4-4952-97c7-7a04c714b48c-original-pull-secret\") pod \"global-pull-secret-syncer-4vtdc\" (UID: \"d17b7754-d1d4-4952-97c7-7a04c714b48c\") " pod="kube-system/global-pull-secret-syncer-4vtdc"
Oct 30 21:52:21.678562 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.678445    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8806f1e1-0057-4738-8fbc-7e503e1f91ca-metrics-certs\") pod \"network-metrics-daemon-wpdr9\" (UID: \"8806f1e1-0057-4738-8fbc-7e503e1f91ca\") " pod="openshift-multus/network-metrics-daemon-wpdr9"
Oct 30 21:52:21.678562 ip-10-0-131-202 kubenswrapper[2578]: E1030 21:52:21.678517    2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 30 21:52:21.678562 ip-10-0-131-202 kubenswrapper[2578]: E1030 21:52:21.678551    2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 21:52:21.678666 ip-10-0-131-202 kubenswrapper[2578]: E1030 21:52:21.678587    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d17b7754-d1d4-4952-97c7-7a04c714b48c-original-pull-secret podName:d17b7754-d1d4-4952-97c7-7a04c714b48c nodeName:}" failed. No retries permitted until 2026-10-30 21:52:22.678568735 +0000 UTC m=+3.318268939 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d17b7754-d1d4-4952-97c7-7a04c714b48c-original-pull-secret") pod "global-pull-secret-syncer-4vtdc" (UID: "d17b7754-d1d4-4952-97c7-7a04c714b48c") : object "kube-system"/"original-pull-secret" not registered
Oct 30 21:52:21.678666 ip-10-0-131-202 kubenswrapper[2578]: E1030 21:52:21.678603    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8806f1e1-0057-4738-8fbc-7e503e1f91ca-metrics-certs podName:8806f1e1-0057-4738-8fbc-7e503e1f91ca nodeName:}" failed. No retries permitted until 2026-10-30 21:52:22.678596045 +0000 UTC m=+3.318296254 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8806f1e1-0057-4738-8fbc-7e503e1f91ca-metrics-certs") pod "network-metrics-daemon-wpdr9" (UID: "8806f1e1-0057-4738-8fbc-7e503e1f91ca") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 21:52:21.779051 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.779015    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6j99v\" (UniqueName: \"kubernetes.io/projected/7be00c31-281f-4d1c-8e12-1bf74bbe7cbd-kube-api-access-6j99v\") pod \"network-check-target-slfvl\" (UID: \"7be00c31-281f-4d1c-8e12-1bf74bbe7cbd\") " pod="openshift-network-diagnostics/network-check-target-slfvl"
Oct 30 21:52:21.779781 ip-10-0-131-202 kubenswrapper[2578]: E1030 21:52:21.779132    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 30 21:52:21.779781 ip-10-0-131-202 kubenswrapper[2578]: E1030 21:52:21.779166    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 30 21:52:21.779781 ip-10-0-131-202 kubenswrapper[2578]: E1030 21:52:21.779177    2578 projected.go:196] Error preparing data for projected volume kube-api-access-6j99v for pod openshift-network-diagnostics/network-check-target-slfvl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 21:52:21.779781 ip-10-0-131-202 kubenswrapper[2578]: E1030 21:52:21.779225    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7be00c31-281f-4d1c-8e12-1bf74bbe7cbd-kube-api-access-6j99v podName:7be00c31-281f-4d1c-8e12-1bf74bbe7cbd nodeName:}" failed. No retries permitted until 2026-10-30 21:52:22.779210318 +0000 UTC m=+3.418910520 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-6j99v" (UniqueName: "kubernetes.io/projected/7be00c31-281f-4d1c-8e12-1bf74bbe7cbd-kube-api-access-6j99v") pod "network-check-target-slfvl" (UID: "7be00c31-281f-4d1c-8e12-1bf74bbe7cbd") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 21:52:21.916728 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.916630    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-hf5t4" event={"ID":"b1a02001-9121-47bd-ae96-b39196ad2357","Type":"ContainerStarted","Data":"3f2b3c7943e258b620f38697047406863aa88efda8d5faada68af4c4e955aeb5"}
Oct 30 21:52:21.917737 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.917714    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-7v7fb" event={"ID":"dffd7a70-b7f3-4e1b-b3ab-74c5a574cab2","Type":"ContainerStarted","Data":"da317ccc84703e328e24540d39e76dcf76ce091d7baad369756dba3796160cfb"}
Oct 30 21:52:21.918910 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.918884    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lhj9s" event={"ID":"bf0e36d0-87bd-4e1e-be40-8a3a0f856987","Type":"ContainerStarted","Data":"437d7c0fb64e2862943d48eb6c7731f8d740ca3387b8624fad4e28e5b29e1511"}
Oct 30 21:52:21.920048 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.920015    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-x7l5l" event={"ID":"25590e1f-99e3-4469-8a69-39cd30317c28","Type":"ContainerStarted","Data":"a9887498d2723b7a9e696b07db4f758f0c7f19897e1b590ceb5739927bce13cb"}
Oct 30 21:52:21.921118 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.921094    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-202.us-west-1.compute.internal" event={"ID":"ec1e59c1a0a508a7c75743ab25eff8ba","Type":"ContainerStarted","Data":"9751c27f53d33b0e6f09e8d175690fc83db3fe0e1f08314102e30aca663fe0c9"}
Oct 30 21:52:21.922151 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.922083    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-9zvbw" event={"ID":"982efb96-3613-4ef2-ac51-ed3f40745a32","Type":"ContainerStarted","Data":"6d0727caaaf3fa2870ab13c51cbab65fbfbb7687ef8ff9096a16e38f970f0243"}
Oct 30 21:52:21.923047 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.923028    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-202.us-west-1.compute.internal" event={"ID":"04337a6f3666f0af07c3d2b702c6e379","Type":"ContainerStarted","Data":"693c46ffe0fe055558dabec72ff633a79b91a6e9ec57f3e27c1c31178984aaa7"}
Oct 30 21:52:21.924056 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:21.924037    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vpbvr" event={"ID":"c590d5fe-57de-4905-90fe-8527b215ce94","Type":"ContainerStarted","Data":"6e596f1f222be642b03bca48eb0148c607deb4bab720ed68a09e44e31f2c0dea"}
Oct 30 21:52:22.136387 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:22.136323    2578 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod164bafd8_ff17_46a2_afc0_dd01fa1aaeb6.slice/crio-705914d7dce2552b00ce3ab62bd974e25578a0cc87f776136f313b6d4f04b10e WatchSource:0}: Error finding container 705914d7dce2552b00ce3ab62bd974e25578a0cc87f776136f313b6d4f04b10e: Status 404 returned error can't find the container with id 705914d7dce2552b00ce3ab62bd974e25578a0cc87f776136f313b6d4f04b10e
Oct 30 21:52:22.147688 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:22.147631    2578 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-10-29 21:47:21 +0000 UTC" deadline="2028-04-05 02:11:27.377471726 +0000 UTC"
Oct 30 21:52:22.147688 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:22.147683    2578 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12532h19m5.229791886s"
Oct 30 21:52:22.177981 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:22.177934    2578 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 21:52:22.187159 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:22.187132    2578 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod386f9569_a5f5_4a9e_beee_3005994186cd.slice/crio-9e1ebfcd253a8ad808556ec9db0d8260ed7f2447b3ad5380483f2d6e13239593 WatchSource:0}: Error finding container 9e1ebfcd253a8ad808556ec9db0d8260ed7f2447b3ad5380483f2d6e13239593: Status 404 returned error can't find the container with id 9e1ebfcd253a8ad808556ec9db0d8260ed7f2447b3ad5380483f2d6e13239593
Oct 30 21:52:22.250774 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:22.250746    2578 reflector.go:446] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 21:52:22.284211 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:22.284184    2578 reflector.go:446] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 21:52:22.349855 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:22.349819    2578 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda4ff4cce_671b_4136_8b63_b22148c0db4e.slice/crio-3970e1be64d4877f260bff03ca6a43749580fdc4c5a49479929398c86576395e WatchSource:0}: Error finding container 3970e1be64d4877f260bff03ca6a43749580fdc4c5a49479929398c86576395e: Status 404 returned error can't find the container with id 3970e1be64d4877f260bff03ca6a43749580fdc4c5a49479929398c86576395e
Oct 30 21:52:22.684664 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:22.684616    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d17b7754-d1d4-4952-97c7-7a04c714b48c-original-pull-secret\") pod \"global-pull-secret-syncer-4vtdc\" (UID: \"d17b7754-d1d4-4952-97c7-7a04c714b48c\") " pod="kube-system/global-pull-secret-syncer-4vtdc"
Oct 30 21:52:22.684866 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:22.684703    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8806f1e1-0057-4738-8fbc-7e503e1f91ca-metrics-certs\") pod \"network-metrics-daemon-wpdr9\" (UID: \"8806f1e1-0057-4738-8fbc-7e503e1f91ca\") " pod="openshift-multus/network-metrics-daemon-wpdr9"
Oct 30 21:52:22.684866 ip-10-0-131-202 kubenswrapper[2578]: E1030 21:52:22.684799    2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 30 21:52:22.684866 ip-10-0-131-202 kubenswrapper[2578]: E1030 21:52:22.684835    2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 21:52:22.684866 ip-10-0-131-202 kubenswrapper[2578]: E1030 21:52:22.684875    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d17b7754-d1d4-4952-97c7-7a04c714b48c-original-pull-secret podName:d17b7754-d1d4-4952-97c7-7a04c714b48c nodeName:}" failed. No retries permitted until 2026-10-30 21:52:24.684854454 +0000 UTC m=+5.324554672 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d17b7754-d1d4-4952-97c7-7a04c714b48c-original-pull-secret") pod "global-pull-secret-syncer-4vtdc" (UID: "d17b7754-d1d4-4952-97c7-7a04c714b48c") : object "kube-system"/"original-pull-secret" not registered
Oct 30 21:52:22.685217 ip-10-0-131-202 kubenswrapper[2578]: E1030 21:52:22.684902    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8806f1e1-0057-4738-8fbc-7e503e1f91ca-metrics-certs podName:8806f1e1-0057-4738-8fbc-7e503e1f91ca nodeName:}" failed. No retries permitted until 2026-10-30 21:52:24.684883268 +0000 UTC m=+5.324583494 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8806f1e1-0057-4738-8fbc-7e503e1f91ca-metrics-certs") pod "network-metrics-daemon-wpdr9" (UID: "8806f1e1-0057-4738-8fbc-7e503e1f91ca") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 21:52:22.785736 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:22.785698    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6j99v\" (UniqueName: \"kubernetes.io/projected/7be00c31-281f-4d1c-8e12-1bf74bbe7cbd-kube-api-access-6j99v\") pod \"network-check-target-slfvl\" (UID: \"7be00c31-281f-4d1c-8e12-1bf74bbe7cbd\") " pod="openshift-network-diagnostics/network-check-target-slfvl"
Oct 30 21:52:22.786285 ip-10-0-131-202 kubenswrapper[2578]: E1030 21:52:22.785876    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 30 21:52:22.786285 ip-10-0-131-202 kubenswrapper[2578]: E1030 21:52:22.785899    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 30 21:52:22.786285 ip-10-0-131-202 kubenswrapper[2578]: E1030 21:52:22.785913    2578 projected.go:196] Error preparing data for projected volume kube-api-access-6j99v for pod openshift-network-diagnostics/network-check-target-slfvl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 21:52:22.786285 ip-10-0-131-202 kubenswrapper[2578]: E1030 21:52:22.785974    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7be00c31-281f-4d1c-8e12-1bf74bbe7cbd-kube-api-access-6j99v podName:7be00c31-281f-4d1c-8e12-1bf74bbe7cbd nodeName:}" failed. No retries permitted until 2026-10-30 21:52:24.785953817 +0000 UTC m=+5.425654037 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-6j99v" (UniqueName: "kubernetes.io/projected/7be00c31-281f-4d1c-8e12-1bf74bbe7cbd-kube-api-access-6j99v") pod "network-check-target-slfvl" (UID: "7be00c31-281f-4d1c-8e12-1bf74bbe7cbd") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 21:52:22.910356 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:22.910321    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-slfvl"
Oct 30 21:52:22.910550 ip-10-0-131-202 kubenswrapper[2578]: E1030 21:52:22.910463    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-slfvl" podUID="7be00c31-281f-4d1c-8e12-1bf74bbe7cbd"
Oct 30 21:52:22.910622 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:22.910557    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wpdr9"
Oct 30 21:52:22.910675 ip-10-0-131-202 kubenswrapper[2578]: E1030 21:52:22.910651    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wpdr9" podUID="8806f1e1-0057-4738-8fbc-7e503e1f91ca"
Oct 30 21:52:22.910737 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:22.910707    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4vtdc"
Oct 30 21:52:22.910791 ip-10-0-131-202 kubenswrapper[2578]: E1030 21:52:22.910774    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4vtdc" podUID="d17b7754-d1d4-4952-97c7-7a04c714b48c"
Oct 30 21:52:22.946698 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:22.946613    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-2hzq7" event={"ID":"a4ff4cce-671b-4136-8b63-b22148c0db4e","Type":"ContainerStarted","Data":"3970e1be64d4877f260bff03ca6a43749580fdc4c5a49479929398c86576395e"}
Oct 30 21:52:22.953421 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:22.953023    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nd9qk" event={"ID":"386f9569-a5f5-4a9e-beee-3005994186cd","Type":"ContainerStarted","Data":"9e1ebfcd253a8ad808556ec9db0d8260ed7f2447b3ad5380483f2d6e13239593"}
Oct 30 21:52:22.973334 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:22.973304    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-f9t6t" event={"ID":"164bafd8-ff17-46a2-afc0-dd01fa1aaeb6","Type":"ContainerStarted","Data":"705914d7dce2552b00ce3ab62bd974e25578a0cc87f776136f313b6d4f04b10e"}
Oct 30 21:52:23.148353 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:23.148293    2578 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-10-29 21:47:21 +0000 UTC" deadline="2028-06-16 13:36:35.292084548 +0000 UTC"
Oct 30 21:52:23.148353 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:23.148335    2578 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14271h44m12.143754448s"
Oct 30 21:52:24.703024 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:24.702940    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8806f1e1-0057-4738-8fbc-7e503e1f91ca-metrics-certs\") pod \"network-metrics-daemon-wpdr9\" (UID: \"8806f1e1-0057-4738-8fbc-7e503e1f91ca\") " pod="openshift-multus/network-metrics-daemon-wpdr9"
Oct 30 21:52:24.703024 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:24.703006    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d17b7754-d1d4-4952-97c7-7a04c714b48c-original-pull-secret\") pod \"global-pull-secret-syncer-4vtdc\" (UID: \"d17b7754-d1d4-4952-97c7-7a04c714b48c\") " pod="kube-system/global-pull-secret-syncer-4vtdc"
Oct 30 21:52:24.703495 ip-10-0-131-202 kubenswrapper[2578]: E1030 21:52:24.703290    2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 21:52:24.703495 ip-10-0-131-202 kubenswrapper[2578]: E1030 21:52:24.703347    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8806f1e1-0057-4738-8fbc-7e503e1f91ca-metrics-certs podName:8806f1e1-0057-4738-8fbc-7e503e1f91ca nodeName:}" failed. No retries permitted until 2026-10-30 21:52:28.703330076 +0000 UTC m=+9.343030292 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8806f1e1-0057-4738-8fbc-7e503e1f91ca-metrics-certs") pod "network-metrics-daemon-wpdr9" (UID: "8806f1e1-0057-4738-8fbc-7e503e1f91ca") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 21:52:24.703897 ip-10-0-131-202 kubenswrapper[2578]: E1030 21:52:24.703875    2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 30 21:52:24.704090 ip-10-0-131-202 kubenswrapper[2578]: E1030 21:52:24.703939    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d17b7754-d1d4-4952-97c7-7a04c714b48c-original-pull-secret podName:d17b7754-d1d4-4952-97c7-7a04c714b48c nodeName:}" failed. No retries permitted until 2026-10-30 21:52:28.703923705 +0000 UTC m=+9.343623913 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d17b7754-d1d4-4952-97c7-7a04c714b48c-original-pull-secret") pod "global-pull-secret-syncer-4vtdc" (UID: "d17b7754-d1d4-4952-97c7-7a04c714b48c") : object "kube-system"/"original-pull-secret" not registered
Oct 30 21:52:24.806138 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:24.803638    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6j99v\" (UniqueName: \"kubernetes.io/projected/7be00c31-281f-4d1c-8e12-1bf74bbe7cbd-kube-api-access-6j99v\") pod \"network-check-target-slfvl\" (UID: \"7be00c31-281f-4d1c-8e12-1bf74bbe7cbd\") " pod="openshift-network-diagnostics/network-check-target-slfvl"
Oct 30 21:52:24.806138 ip-10-0-131-202 kubenswrapper[2578]: E1030 21:52:24.803824    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 30 21:52:24.806138 ip-10-0-131-202 kubenswrapper[2578]: E1030 21:52:24.803848    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 30 21:52:24.806138 ip-10-0-131-202 kubenswrapper[2578]: E1030 21:52:24.803861    2578 projected.go:196] Error preparing data for projected volume kube-api-access-6j99v for pod openshift-network-diagnostics/network-check-target-slfvl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 21:52:24.806138 ip-10-0-131-202 kubenswrapper[2578]: E1030 21:52:24.803919    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7be00c31-281f-4d1c-8e12-1bf74bbe7cbd-kube-api-access-6j99v podName:7be00c31-281f-4d1c-8e12-1bf74bbe7cbd nodeName:}" failed. No retries permitted until 2026-10-30 21:52:28.80389954 +0000 UTC m=+9.443599758 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-6j99v" (UniqueName: "kubernetes.io/projected/7be00c31-281f-4d1c-8e12-1bf74bbe7cbd-kube-api-access-6j99v") pod "network-check-target-slfvl" (UID: "7be00c31-281f-4d1c-8e12-1bf74bbe7cbd") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 21:52:24.910241 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:24.910189    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4vtdc"
Oct 30 21:52:24.910427 ip-10-0-131-202 kubenswrapper[2578]: E1030 21:52:24.910348    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4vtdc" podUID="d17b7754-d1d4-4952-97c7-7a04c714b48c"
Oct 30 21:52:24.910427 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:24.910373    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wpdr9"
Oct 30 21:52:24.910555 ip-10-0-131-202 kubenswrapper[2578]: E1030 21:52:24.910494    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wpdr9" podUID="8806f1e1-0057-4738-8fbc-7e503e1f91ca"
Oct 30 21:52:24.910555 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:24.910523    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-slfvl"
Oct 30 21:52:24.910656 ip-10-0-131-202 kubenswrapper[2578]: E1030 21:52:24.910595    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-slfvl" podUID="7be00c31-281f-4d1c-8e12-1bf74bbe7cbd"
Oct 30 21:52:26.009347 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:26.008974    2578 generic.go:356] "Generic (PLEG): container finished" podID="c590d5fe-57de-4905-90fe-8527b215ce94" containerID="1b3b49caae15fa36b2e5d6780f25b557570358db3a35a3f1bec2561271156924" exitCode=0
Oct 30 21:52:26.009844 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:26.009447    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vpbvr" event={"ID":"c590d5fe-57de-4905-90fe-8527b215ce94","Type":"ContainerDied","Data":"1b3b49caae15fa36b2e5d6780f25b557570358db3a35a3f1bec2561271156924"}
Oct 30 21:52:26.910510 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:26.910466    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-slfvl"
Oct 30 21:52:26.910681 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:26.910522    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4vtdc"
Oct 30 21:52:26.910681 ip-10-0-131-202 kubenswrapper[2578]: E1030 21:52:26.910638    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-slfvl" podUID="7be00c31-281f-4d1c-8e12-1bf74bbe7cbd"
Oct 30 21:52:26.910681 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:26.910496    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wpdr9"
Oct 30 21:52:26.910861 ip-10-0-131-202 kubenswrapper[2578]: E1030 21:52:26.910760    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4vtdc" podUID="d17b7754-d1d4-4952-97c7-7a04c714b48c"
Oct 30 21:52:26.910923 ip-10-0-131-202 kubenswrapper[2578]: E1030 21:52:26.910853    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wpdr9" podUID="8806f1e1-0057-4738-8fbc-7e503e1f91ca"
Oct 30 21:52:27.014584 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:27.014539    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-x7l5l" event={"ID":"25590e1f-99e3-4469-8a69-39cd30317c28","Type":"ContainerStarted","Data":"190bba81c09a44b1ad11c72911b70814fbc4fbec33fd65f57a03101afb78ff01"}
Oct 30 21:52:27.020153 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:27.020098    2578 generic.go:356] "Generic (PLEG): container finished" podID="ec1e59c1a0a508a7c75743ab25eff8ba" containerID="25993f737081014a0fe5f6981adc9bdc77e6a44d7aab47aa22ecfffbf9c40ee5" exitCode=0
Oct 30 21:52:27.020300 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:27.020165    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-202.us-west-1.compute.internal" event={"ID":"ec1e59c1a0a508a7c75743ab25eff8ba","Type":"ContainerDied","Data":"25993f737081014a0fe5f6981adc9bdc77e6a44d7aab47aa22ecfffbf9c40ee5"}
Oct 30 21:52:27.261231 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:27.260152    2578 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/konnectivity-agent-x7l5l" podStartSLOduration=2.194289448 podStartE2EDuration="7.260129756s" totalImagesPullingTime="5.065840308s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:52:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:52:27.137377336 +0000 UTC m=+7.777077561" watchObservedRunningTime="2026-10-30 21:52:27.260129756 +0000 UTC m=+7.899829985"
Oct 30 21:52:28.026985 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:28.026224    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-202.us-west-1.compute.internal" event={"ID":"ec1e59c1a0a508a7c75743ab25eff8ba","Type":"ContainerStarted","Data":"d337aed051dcee497300943e642bba3c14228bb4e06427a412d286f8e500ae25"}
Oct 30 21:52:28.033123 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:28.033039    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-202.us-west-1.compute.internal" event={"ID":"04337a6f3666f0af07c3d2b702c6e379","Type":"ContainerStarted","Data":"ffb6e25f287382f2c628231ad8a3ad3fed2eaba2281db36a37cdce8064bebbc6"}
Oct 30 21:52:28.150881 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:28.150300    2578 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-202.us-west-1.compute.internal" podStartSLOduration=8.150279925 podStartE2EDuration="8.150279925s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 21:52:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:52:28.149521621 +0000 UTC m=+8.789221846" watchObservedRunningTime="2026-10-30 21:52:28.150279925 +0000 UTC m=+8.789980158"
Oct 30 21:52:28.507560 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:28.507525    2578 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-x7l5l"
Oct 30 21:52:28.508310 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:28.508287    2578 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-x7l5l"
Oct 30 21:52:28.634165 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:28.634110    2578 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-202.us-west-1.compute.internal" podStartSLOduration=8.634093767 podStartE2EDuration="8.634093767s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 21:52:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:52:28.272892048 +0000 UTC m=+8.912592276" watchObservedRunningTime="2026-10-30 21:52:28.634093767 +0000 UTC m=+9.273793995"
Oct 30 21:52:28.741408 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:28.741378    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d17b7754-d1d4-4952-97c7-7a04c714b48c-original-pull-secret\") pod \"global-pull-secret-syncer-4vtdc\" (UID: \"d17b7754-d1d4-4952-97c7-7a04c714b48c\") " pod="kube-system/global-pull-secret-syncer-4vtdc"
Oct 30 21:52:28.741553 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:28.741450    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8806f1e1-0057-4738-8fbc-7e503e1f91ca-metrics-certs\") pod \"network-metrics-daemon-wpdr9\" (UID: \"8806f1e1-0057-4738-8fbc-7e503e1f91ca\") " pod="openshift-multus/network-metrics-daemon-wpdr9"
Oct 30 21:52:28.741623 ip-10-0-131-202 kubenswrapper[2578]: E1030 21:52:28.741594    2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 21:52:28.741683 ip-10-0-131-202 kubenswrapper[2578]: E1030 21:52:28.741660    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8806f1e1-0057-4738-8fbc-7e503e1f91ca-metrics-certs podName:8806f1e1-0057-4738-8fbc-7e503e1f91ca nodeName:}" failed. No retries permitted until 2026-10-30 21:52:36.74163878 +0000 UTC m=+17.381339000 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8806f1e1-0057-4738-8fbc-7e503e1f91ca-metrics-certs") pod "network-metrics-daemon-wpdr9" (UID: "8806f1e1-0057-4738-8fbc-7e503e1f91ca") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 21:52:28.741744 ip-10-0-131-202 kubenswrapper[2578]: E1030 21:52:28.741732    2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 30 21:52:28.741796 ip-10-0-131-202 kubenswrapper[2578]: E1030 21:52:28.741770    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d17b7754-d1d4-4952-97c7-7a04c714b48c-original-pull-secret podName:d17b7754-d1d4-4952-97c7-7a04c714b48c nodeName:}" failed. No retries permitted until 2026-10-30 21:52:36.741757517 +0000 UTC m=+17.381457735 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d17b7754-d1d4-4952-97c7-7a04c714b48c-original-pull-secret") pod "global-pull-secret-syncer-4vtdc" (UID: "d17b7754-d1d4-4952-97c7-7a04c714b48c") : object "kube-system"/"original-pull-secret" not registered
Oct 30 21:52:28.841808 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:28.841770    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6j99v\" (UniqueName: \"kubernetes.io/projected/7be00c31-281f-4d1c-8e12-1bf74bbe7cbd-kube-api-access-6j99v\") pod \"network-check-target-slfvl\" (UID: \"7be00c31-281f-4d1c-8e12-1bf74bbe7cbd\") " pod="openshift-network-diagnostics/network-check-target-slfvl"
Oct 30 21:52:28.841998 ip-10-0-131-202 kubenswrapper[2578]: E1030 21:52:28.841987    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 30 21:52:28.842060 ip-10-0-131-202 kubenswrapper[2578]: E1030 21:52:28.842008    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 30 21:52:28.842060 ip-10-0-131-202 kubenswrapper[2578]: E1030 21:52:28.842020    2578 projected.go:196] Error preparing data for projected volume kube-api-access-6j99v for pod openshift-network-diagnostics/network-check-target-slfvl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 21:52:28.842225 ip-10-0-131-202 kubenswrapper[2578]: E1030 21:52:28.842081    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7be00c31-281f-4d1c-8e12-1bf74bbe7cbd-kube-api-access-6j99v podName:7be00c31-281f-4d1c-8e12-1bf74bbe7cbd nodeName:}" failed. No retries permitted until 2026-10-30 21:52:36.84206182 +0000 UTC m=+17.481762039 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-6j99v" (UniqueName: "kubernetes.io/projected/7be00c31-281f-4d1c-8e12-1bf74bbe7cbd-kube-api-access-6j99v") pod "network-check-target-slfvl" (UID: "7be00c31-281f-4d1c-8e12-1bf74bbe7cbd") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 21:52:28.910910 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:28.910868    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-slfvl"
Oct 30 21:52:28.911096 ip-10-0-131-202 kubenswrapper[2578]: E1030 21:52:28.911017    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-slfvl" podUID="7be00c31-281f-4d1c-8e12-1bf74bbe7cbd"
Oct 30 21:52:28.911162 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:28.911106    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wpdr9"
Oct 30 21:52:28.911268 ip-10-0-131-202 kubenswrapper[2578]: E1030 21:52:28.911242    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wpdr9" podUID="8806f1e1-0057-4738-8fbc-7e503e1f91ca"
Oct 30 21:52:28.911407 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:28.911310    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4vtdc"
Oct 30 21:52:28.911519 ip-10-0-131-202 kubenswrapper[2578]: E1030 21:52:28.911489    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4vtdc" podUID="d17b7754-d1d4-4952-97c7-7a04c714b48c"
Oct 30 21:52:29.039657 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:29.039623    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-f9t6t" event={"ID":"164bafd8-ff17-46a2-afc0-dd01fa1aaeb6","Type":"ContainerStarted","Data":"288ef9f4c4c4c05a16ffe6c767015067c5b79b850c3b6fb866d277786075d4e5"}
Oct 30 21:52:30.046887 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:30.046855    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nd9qk" event={"ID":"386f9569-a5f5-4a9e-beee-3005994186cd","Type":"ContainerStarted","Data":"ac3f04aeee91a02edc49b048c9aca73a714c2410d67e10749b1cef5570df7448"}
Oct 30 21:52:30.047440 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:30.046947    2578 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Oct 30 21:52:30.911921 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:30.911173    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4vtdc"
Oct 30 21:52:30.911921 ip-10-0-131-202 kubenswrapper[2578]: E1030 21:52:30.911326    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4vtdc" podUID="d17b7754-d1d4-4952-97c7-7a04c714b48c"
Oct 30 21:52:30.911921 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:30.911642    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-slfvl"
Oct 30 21:52:30.911921 ip-10-0-131-202 kubenswrapper[2578]: E1030 21:52:30.911720    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-slfvl" podUID="7be00c31-281f-4d1c-8e12-1bf74bbe7cbd"
Oct 30 21:52:30.911921 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:30.911778    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wpdr9"
Oct 30 21:52:30.911921 ip-10-0-131-202 kubenswrapper[2578]: E1030 21:52:30.911888    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wpdr9" podUID="8806f1e1-0057-4738-8fbc-7e503e1f91ca"
Oct 30 21:52:31.051221 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:31.051189    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-7v7fb" event={"ID":"dffd7a70-b7f3-4e1b-b3ab-74c5a574cab2","Type":"ContainerStarted","Data":"cc94e1dd6d4ae16232f1d6acc7bf87fe139479c824f6b2b458924e4c4258a6be"}
Oct 30 21:52:31.054021 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:31.053987    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-hf5t4" event={"ID":"b1a02001-9121-47bd-ae96-b39196ad2357","Type":"ContainerStarted","Data":"6a5bece56381c9002cd3cc43feed05904ed2aabd3a8ba6de06ae12057c087c2a"}
Oct 30 21:52:31.185441 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:31.185383    2578 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-image-registry/node-ca-f9t6t" podStartSLOduration=4.713871064 podStartE2EDuration="11.185364066s" totalImagesPullingTime="6.471493002s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 21:52:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:52:29.16395191 +0000 UTC m=+9.803652139" watchObservedRunningTime="2026-10-30 21:52:31.185364066 +0000 UTC m=+11.825064289"
Oct 30 21:52:31.305450 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:31.305377    2578 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-dns/node-resolver-7v7fb" podStartSLOduration=2.013301407 podStartE2EDuration="11.305356006s" totalImagesPullingTime="9.292054599s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 21:52:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:52:31.185655532 +0000 UTC m=+11.825355755" watchObservedRunningTime="2026-10-30 21:52:31.305356006 +0000 UTC m=+11.945056229"
Oct 30 21:52:31.305655 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:31.305621    2578 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-multus/multus-hf5t4" podStartSLOduration=1.8241661489999998 podStartE2EDuration="11.305614158s" totalImagesPullingTime="9.481448009s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 21:52:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:52:31.305204254 +0000 UTC m=+11.944904477" watchObservedRunningTime="2026-10-30 21:52:31.305614158 +0000 UTC m=+11.945314380"
Oct 30 21:52:31.366851 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:31.366798    2578 plugin_watcher.go:201] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Oct 30 21:52:31.841682 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:31.841521    2578 reconciler.go:167] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-10-30T21:52:31.366818954Z","UUID":"95300e7e-f331-4e45-a954-cd66a2a35da0","Handler":null,"Name":"","Endpoint":""}
Oct 30 21:52:31.844533 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:31.844507    2578 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Oct 30 21:52:31.844533 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:31.844542    2578 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Oct 30 21:52:32.058301 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:32.058262    2578 generic.go:356] "Generic (PLEG): container finished" podID="c590d5fe-57de-4905-90fe-8527b215ce94" containerID="ac58b695732aac4d4740d32ea93510c8c01d7aa4c2c7a97e262a4de55cec5f97" exitCode=0
Oct 30 21:52:32.058721 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:32.058351    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vpbvr" event={"ID":"c590d5fe-57de-4905-90fe-8527b215ce94","Type":"ContainerDied","Data":"ac58b695732aac4d4740d32ea93510c8c01d7aa4c2c7a97e262a4de55cec5f97"}
Oct 30 21:52:32.062167 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:32.062139    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nd9qk" event={"ID":"386f9569-a5f5-4a9e-beee-3005994186cd","Type":"ContainerStarted","Data":"a0389f66dbe3b8dec180cad27093836d476f5656f75b6a339b818cfb3f33fbdc"}
Oct 30 21:52:32.910684 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:32.910648    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-slfvl"
Oct 30 21:52:32.910826 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:32.910698    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wpdr9"
Oct 30 21:52:32.910826 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:32.910795    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4vtdc"
Oct 30 21:52:32.910826 ip-10-0-131-202 kubenswrapper[2578]: E1030 21:52:32.910799    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-slfvl" podUID="7be00c31-281f-4d1c-8e12-1bf74bbe7cbd"
Oct 30 21:52:32.910988 ip-10-0-131-202 kubenswrapper[2578]: E1030 21:52:32.910927    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4vtdc" podUID="d17b7754-d1d4-4952-97c7-7a04c714b48c"
Oct 30 21:52:32.911051 ip-10-0-131-202 kubenswrapper[2578]: E1030 21:52:32.911027    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wpdr9" podUID="8806f1e1-0057-4738-8fbc-7e503e1f91ca"
Oct 30 21:52:33.065498 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:33.065096    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-2hzq7" event={"ID":"a4ff4cce-671b-4136-8b63-b22148c0db4e","Type":"ContainerStarted","Data":"d3bba8399e857101a03e536ac11497937109ac263b51aea43efe2cb8ef852deb"}
Oct 30 21:52:33.182202 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:33.182082    2578 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-2hzq7" podStartSLOduration=4.149058378 podStartE2EDuration="13.182066374s" totalImagesPullingTime="9.033007996s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 21:52:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:52:33.181786054 +0000 UTC m=+13.821486277" watchObservedRunningTime="2026-10-30 21:52:33.182066374 +0000 UTC m=+13.821766598"
Oct 30 21:52:33.421613 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:33.421579    2578 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-x7l5l"
Oct 30 21:52:33.421761 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:33.421714    2578 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Oct 30 21:52:33.422286 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:33.422265    2578 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-x7l5l"
Oct 30 21:52:34.070350 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:34.070268    2578 generic.go:356] "Generic (PLEG): container finished" podID="c590d5fe-57de-4905-90fe-8527b215ce94" containerID="c17bf4a32f01280e4ed086fdb68fa78e9a0e961dd25f50f6c5be54d57c1e0a20" exitCode=0
Oct 30 21:52:34.070938 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:34.070377    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vpbvr" event={"ID":"c590d5fe-57de-4905-90fe-8527b215ce94","Type":"ContainerDied","Data":"c17bf4a32f01280e4ed086fdb68fa78e9a0e961dd25f50f6c5be54d57c1e0a20"}
Oct 30 21:52:34.073208 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:34.073164    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nd9qk" event={"ID":"386f9569-a5f5-4a9e-beee-3005994186cd","Type":"ContainerStarted","Data":"a32f487e0e8e01933911dde59fed83cb3b45e30479c017faa5df5cf7dd1c0421"}
Oct 30 21:52:34.074729 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:34.074706    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-9zvbw" event={"ID":"982efb96-3613-4ef2-ac51-ed3f40745a32","Type":"ContainerStarted","Data":"f893d5a3b649d2d273c9d2003e45fad1061affc50cdf7429d12d64c7f8567765"}
Oct 30 21:52:34.307074 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:34.307019    2578 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nd9qk" podStartSLOduration=3.342103727 podStartE2EDuration="14.307004921s" totalImagesPullingTime="10.964901194s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 21:52:20 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:52:34.30603882 +0000 UTC m=+14.945739044" watchObservedRunningTime="2026-10-30 21:52:34.307004921 +0000 UTC m=+14.946705142"
Oct 30 21:52:34.421760 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:34.421647    2578 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-9zvbw" podStartSLOduration=2.335870169 podStartE2EDuration="14.421628783s" totalImagesPullingTime="12.085758614s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 21:52:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:52:34.421548546 +0000 UTC m=+15.061248770" watchObservedRunningTime="2026-10-30 21:52:34.421628783 +0000 UTC m=+15.061329007"
Oct 30 21:52:34.910719 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:34.910683    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4vtdc"
Oct 30 21:52:34.910719 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:34.910711    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-slfvl"
Oct 30 21:52:34.910935 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:34.910875    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wpdr9"
Oct 30 21:52:34.910935 ip-10-0-131-202 kubenswrapper[2578]: E1030 21:52:34.910882    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4vtdc" podUID="d17b7754-d1d4-4952-97c7-7a04c714b48c"
Oct 30 21:52:34.911016 ip-10-0-131-202 kubenswrapper[2578]: E1030 21:52:34.910959    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-slfvl" podUID="7be00c31-281f-4d1c-8e12-1bf74bbe7cbd"
Oct 30 21:52:34.911074 ip-10-0-131-202 kubenswrapper[2578]: E1030 21:52:34.911047    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wpdr9" podUID="8806f1e1-0057-4738-8fbc-7e503e1f91ca"
Oct 30 21:52:36.082292 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:36.082258    2578 generic.go:356] "Generic (PLEG): container finished" podID="c590d5fe-57de-4905-90fe-8527b215ce94" containerID="0f9580bee00ff200dd76780b6aaeb6ae57fa603d544e06a7d591e53f6ce07cfe" exitCode=0
Oct 30 21:52:36.082962 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:36.082318    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vpbvr" event={"ID":"c590d5fe-57de-4905-90fe-8527b215ce94","Type":"ContainerDied","Data":"0f9580bee00ff200dd76780b6aaeb6ae57fa603d544e06a7d591e53f6ce07cfe"}
Oct 30 21:52:36.809428 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:36.809391    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8806f1e1-0057-4738-8fbc-7e503e1f91ca-metrics-certs\") pod \"network-metrics-daemon-wpdr9\" (UID: \"8806f1e1-0057-4738-8fbc-7e503e1f91ca\") " pod="openshift-multus/network-metrics-daemon-wpdr9"
Oct 30 21:52:36.809635 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:36.809466    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d17b7754-d1d4-4952-97c7-7a04c714b48c-original-pull-secret\") pod \"global-pull-secret-syncer-4vtdc\" (UID: \"d17b7754-d1d4-4952-97c7-7a04c714b48c\") " pod="kube-system/global-pull-secret-syncer-4vtdc"
Oct 30 21:52:36.809635 ip-10-0-131-202 kubenswrapper[2578]: E1030 21:52:36.809562    2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 21:52:36.809635 ip-10-0-131-202 kubenswrapper[2578]: E1030 21:52:36.809600    2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 30 21:52:36.809635 ip-10-0-131-202 kubenswrapper[2578]: E1030 21:52:36.809632    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8806f1e1-0057-4738-8fbc-7e503e1f91ca-metrics-certs podName:8806f1e1-0057-4738-8fbc-7e503e1f91ca nodeName:}" failed. No retries permitted until 2026-10-30 21:52:52.809613203 +0000 UTC m=+33.449313420 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8806f1e1-0057-4738-8fbc-7e503e1f91ca-metrics-certs") pod "network-metrics-daemon-wpdr9" (UID: "8806f1e1-0057-4738-8fbc-7e503e1f91ca") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 21:52:36.809795 ip-10-0-131-202 kubenswrapper[2578]: E1030 21:52:36.809654    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d17b7754-d1d4-4952-97c7-7a04c714b48c-original-pull-secret podName:d17b7754-d1d4-4952-97c7-7a04c714b48c nodeName:}" failed. No retries permitted until 2026-10-30 21:52:52.809639701 +0000 UTC m=+33.449339915 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d17b7754-d1d4-4952-97c7-7a04c714b48c-original-pull-secret") pod "global-pull-secret-syncer-4vtdc" (UID: "d17b7754-d1d4-4952-97c7-7a04c714b48c") : object "kube-system"/"original-pull-secret" not registered
Oct 30 21:52:36.910474 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:36.910443    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-slfvl"
Oct 30 21:52:36.910637 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:36.910507    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6j99v\" (UniqueName: \"kubernetes.io/projected/7be00c31-281f-4d1c-8e12-1bf74bbe7cbd-kube-api-access-6j99v\") pod \"network-check-target-slfvl\" (UID: \"7be00c31-281f-4d1c-8e12-1bf74bbe7cbd\") " pod="openshift-network-diagnostics/network-check-target-slfvl"
Oct 30 21:52:36.910637 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:36.910443    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4vtdc"
Oct 30 21:52:36.910637 ip-10-0-131-202 kubenswrapper[2578]: E1030 21:52:36.910563    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-slfvl" podUID="7be00c31-281f-4d1c-8e12-1bf74bbe7cbd"
Oct 30 21:52:36.910637 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:36.910444    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wpdr9"
Oct 30 21:52:36.910792 ip-10-0-131-202 kubenswrapper[2578]: E1030 21:52:36.910636    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4vtdc" podUID="d17b7754-d1d4-4952-97c7-7a04c714b48c"
Oct 30 21:52:36.910792 ip-10-0-131-202 kubenswrapper[2578]: E1030 21:52:36.910659    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 30 21:52:36.910792 ip-10-0-131-202 kubenswrapper[2578]: E1030 21:52:36.910677    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 30 21:52:36.910792 ip-10-0-131-202 kubenswrapper[2578]: E1030 21:52:36.910687    2578 projected.go:196] Error preparing data for projected volume kube-api-access-6j99v for pod openshift-network-diagnostics/network-check-target-slfvl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 21:52:36.910792 ip-10-0-131-202 kubenswrapper[2578]: E1030 21:52:36.910731    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7be00c31-281f-4d1c-8e12-1bf74bbe7cbd-kube-api-access-6j99v podName:7be00c31-281f-4d1c-8e12-1bf74bbe7cbd nodeName:}" failed. No retries permitted until 2026-10-30 21:52:52.910714339 +0000 UTC m=+33.550414557 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-6j99v" (UniqueName: "kubernetes.io/projected/7be00c31-281f-4d1c-8e12-1bf74bbe7cbd-kube-api-access-6j99v") pod "network-check-target-slfvl" (UID: "7be00c31-281f-4d1c-8e12-1bf74bbe7cbd") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 21:52:36.910792 ip-10-0-131-202 kubenswrapper[2578]: E1030 21:52:36.910723    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wpdr9" podUID="8806f1e1-0057-4738-8fbc-7e503e1f91ca"
Oct 30 21:52:38.088612 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:38.088411    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lhj9s" event={"ID":"bf0e36d0-87bd-4e1e-be40-8a3a0f856987","Type":"ContainerStarted","Data":"2864f409dc6c3b1962eb466d6c5e32882344ec02fe106bc46078bc4071c9e522"}
Oct 30 21:52:38.088988 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:38.088878    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lhj9s" event={"ID":"bf0e36d0-87bd-4e1e-be40-8a3a0f856987","Type":"ContainerStarted","Data":"0ba8eb8bf6e09ac51d6273972beca43c22742355e40c09406f272032dd8888c5"}
Oct 30 21:52:38.910566 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:38.910391    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wpdr9"
Oct 30 21:52:38.910756 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:38.910425    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4vtdc"
Oct 30 21:52:38.910756 ip-10-0-131-202 kubenswrapper[2578]: E1030 21:52:38.910661    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wpdr9" podUID="8806f1e1-0057-4738-8fbc-7e503e1f91ca"
Oct 30 21:52:38.910756 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:38.910425    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-slfvl"
Oct 30 21:52:38.910756 ip-10-0-131-202 kubenswrapper[2578]: E1030 21:52:38.910696    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4vtdc" podUID="d17b7754-d1d4-4952-97c7-7a04c714b48c"
Oct 30 21:52:38.910756 ip-10-0-131-202 kubenswrapper[2578]: E1030 21:52:38.910745    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-slfvl" podUID="7be00c31-281f-4d1c-8e12-1bf74bbe7cbd"
Oct 30 21:52:39.093028 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:39.092995    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lhj9s" event={"ID":"bf0e36d0-87bd-4e1e-be40-8a3a0f856987","Type":"ContainerStarted","Data":"605fdc328f6688c18bd6131f8739db7ca9e6be654358e51badf895b5360c18e2"}
Oct 30 21:52:39.093422 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:39.093062    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lhj9s" event={"ID":"bf0e36d0-87bd-4e1e-be40-8a3a0f856987","Type":"ContainerStarted","Data":"0a0b03469f0f413e694f394575e7d73e7fa89381807fd0d1c30adc5c7c99b648"}
Oct 30 21:52:39.093422 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:39.093104    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lhj9s" event={"ID":"bf0e36d0-87bd-4e1e-be40-8a3a0f856987","Type":"ContainerStarted","Data":"92e5b71c7e9b4c5ad45dcc7a1d05e3a1895af512af1b1d4d6089f3eb3f2506b3"}
Oct 30 21:52:39.093422 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:39.093161    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lhj9s" event={"ID":"bf0e36d0-87bd-4e1e-be40-8a3a0f856987","Type":"ContainerStarted","Data":"7f2ecc82ed1f09518bd84119ec7532ee3f6d4ec96b23e092239cc1a238f35079"}
Oct 30 21:52:39.095646 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:39.095624    2578 generic.go:356] "Generic (PLEG): container finished" podID="c590d5fe-57de-4905-90fe-8527b215ce94" containerID="0d21058138ab1ee3fa62a5259ef5c21d8e805bf8a90a24ded701fa3f4af95bca" exitCode=0
Oct 30 21:52:39.095727 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:39.095656    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vpbvr" event={"ID":"c590d5fe-57de-4905-90fe-8527b215ce94","Type":"ContainerDied","Data":"0d21058138ab1ee3fa62a5259ef5c21d8e805bf8a90a24ded701fa3f4af95bca"}
Oct 30 21:52:40.100649 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:40.100611    2578 generic.go:356] "Generic (PLEG): container finished" podID="c590d5fe-57de-4905-90fe-8527b215ce94" containerID="bb3e7be2868bb3748f42810e1a854f57d1614e6e98a15ea8282ec5d345c83c85" exitCode=0
Oct 30 21:52:40.101170 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:40.100675    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vpbvr" event={"ID":"c590d5fe-57de-4905-90fe-8527b215ce94","Type":"ContainerDied","Data":"bb3e7be2868bb3748f42810e1a854f57d1614e6e98a15ea8282ec5d345c83c85"}
Oct 30 21:52:40.911081 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:40.911046    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wpdr9"
Oct 30 21:52:40.911081 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:40.911083    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-slfvl"
Oct 30 21:52:40.911368 ip-10-0-131-202 kubenswrapper[2578]: E1030 21:52:40.911194    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wpdr9" podUID="8806f1e1-0057-4738-8fbc-7e503e1f91ca"
Oct 30 21:52:40.911368 ip-10-0-131-202 kubenswrapper[2578]: E1030 21:52:40.911244    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-slfvl" podUID="7be00c31-281f-4d1c-8e12-1bf74bbe7cbd"
Oct 30 21:52:40.911368 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:40.911285    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4vtdc"
Oct 30 21:52:40.911368 ip-10-0-131-202 kubenswrapper[2578]: E1030 21:52:40.911347    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4vtdc" podUID="d17b7754-d1d4-4952-97c7-7a04c714b48c"
Oct 30 21:52:41.105900 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:41.105866    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lhj9s" event={"ID":"bf0e36d0-87bd-4e1e-be40-8a3a0f856987","Type":"ContainerStarted","Data":"ccdc0c284be7f8056a9d242153bf9c21b6ac803573cdf698c0124ebed2201f19"}
Oct 30 21:52:41.109052 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:41.109029    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vpbvr" event={"ID":"c590d5fe-57de-4905-90fe-8527b215ce94","Type":"ContainerStarted","Data":"4e049460dd6fe0932bf7da0dc5ea6371b597db0b5f46f033af3d97d648024638"}
Oct 30 21:52:41.230655 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:41.230616    2578 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-vpbvr" podStartSLOduration=7.564127146 podStartE2EDuration="21.230603022s" totalImagesPullingTime="13.666475876s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 21:52:20 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:52:41.230418287 +0000 UTC m=+21.870118508" watchObservedRunningTime="2026-10-30 21:52:41.230603022 +0000 UTC m=+21.870303245"
Oct 30 21:52:42.910915 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:42.910884    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wpdr9"
Oct 30 21:52:42.911472 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:42.910917    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4vtdc"
Oct 30 21:52:42.911472 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:42.911001    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-slfvl"
Oct 30 21:52:42.911472 ip-10-0-131-202 kubenswrapper[2578]: E1030 21:52:42.911005    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wpdr9" podUID="8806f1e1-0057-4738-8fbc-7e503e1f91ca"
Oct 30 21:52:42.911472 ip-10-0-131-202 kubenswrapper[2578]: E1030 21:52:42.911076    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4vtdc" podUID="d17b7754-d1d4-4952-97c7-7a04c714b48c"
Oct 30 21:52:42.911472 ip-10-0-131-202 kubenswrapper[2578]: E1030 21:52:42.911203    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-slfvl" podUID="7be00c31-281f-4d1c-8e12-1bf74bbe7cbd"
Oct 30 21:52:43.116844 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:43.116815    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lhj9s" event={"ID":"bf0e36d0-87bd-4e1e-be40-8a3a0f856987","Type":"ContainerStarted","Data":"ae74ba8b901a740c1f98ffefadcca08e50f8b1a49b9c64cb2367a59224995f38"}
Oct 30 21:52:43.117047 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:43.117026    2578 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-lhj9s"
Oct 30 21:52:43.132892 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:43.132870    2578 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-lhj9s"
Oct 30 21:52:43.398978 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:43.398780    2578 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-lhj9s" podStartSLOduration=6.876458891 podStartE2EDuration="23.398767452s" totalImagesPullingTime="16.522308561s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 21:52:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:52:43.349289997 +0000 UTC m=+23.988990218" watchObservedRunningTime="2026-10-30 21:52:43.398767452 +0000 UTC m=+24.038467674"
Oct 30 21:52:44.122468 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:44.122443    2578 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-lhj9s"
Oct 30 21:52:44.122932 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:44.122512    2578 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-lhj9s"
Oct 30 21:52:44.139330 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:44.139308    2578 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-lhj9s"
Oct 30 21:52:44.910490 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:44.910460    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4vtdc"
Oct 30 21:52:44.910660 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:44.910555    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wpdr9"
Oct 30 21:52:44.910660 ip-10-0-131-202 kubenswrapper[2578]: E1030 21:52:44.910563    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4vtdc" podUID="d17b7754-d1d4-4952-97c7-7a04c714b48c"
Oct 30 21:52:44.910660 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:44.910584    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-slfvl"
Oct 30 21:52:44.910759 ip-10-0-131-202 kubenswrapper[2578]: E1030 21:52:44.910665    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wpdr9" podUID="8806f1e1-0057-4738-8fbc-7e503e1f91ca"
Oct 30 21:52:44.910759 ip-10-0-131-202 kubenswrapper[2578]: E1030 21:52:44.910720    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-slfvl" podUID="7be00c31-281f-4d1c-8e12-1bf74bbe7cbd"
Oct 30 21:52:45.235332 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:45.235295    2578 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-4vtdc"]
Oct 30 21:52:45.235760 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:45.235454    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4vtdc"
Oct 30 21:52:45.235760 ip-10-0-131-202 kubenswrapper[2578]: E1030 21:52:45.235563    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4vtdc" podUID="d17b7754-d1d4-4952-97c7-7a04c714b48c"
Oct 30 21:52:45.237949 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:45.237913    2578 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-wpdr9"]
Oct 30 21:52:45.238081 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:45.238066    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wpdr9"
Oct 30 21:52:45.238240 ip-10-0-131-202 kubenswrapper[2578]: E1030 21:52:45.238218    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wpdr9" podUID="8806f1e1-0057-4738-8fbc-7e503e1f91ca"
Oct 30 21:52:45.238560 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:45.238533    2578 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-slfvl"]
Oct 30 21:52:45.238677 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:45.238627    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-slfvl"
Oct 30 21:52:45.238738 ip-10-0-131-202 kubenswrapper[2578]: E1030 21:52:45.238700    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-slfvl" podUID="7be00c31-281f-4d1c-8e12-1bf74bbe7cbd"
Oct 30 21:52:46.911091 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:46.911025    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-slfvl"
Oct 30 21:52:46.911420 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:46.911142    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wpdr9"
Oct 30 21:52:46.911420 ip-10-0-131-202 kubenswrapper[2578]: E1030 21:52:46.911159    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-slfvl" podUID="7be00c31-281f-4d1c-8e12-1bf74bbe7cbd"
Oct 30 21:52:46.911420 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:46.911173    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4vtdc"
Oct 30 21:52:46.911420 ip-10-0-131-202 kubenswrapper[2578]: E1030 21:52:46.911249    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wpdr9" podUID="8806f1e1-0057-4738-8fbc-7e503e1f91ca"
Oct 30 21:52:46.911420 ip-10-0-131-202 kubenswrapper[2578]: E1030 21:52:46.911331    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4vtdc" podUID="d17b7754-d1d4-4952-97c7-7a04c714b48c"
Oct 30 21:52:48.910266 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:48.910237    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-slfvl"
Oct 30 21:52:48.910695 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:48.910323    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wpdr9"
Oct 30 21:52:48.910695 ip-10-0-131-202 kubenswrapper[2578]: E1030 21:52:48.910334    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-slfvl" podUID="7be00c31-281f-4d1c-8e12-1bf74bbe7cbd"
Oct 30 21:52:48.910695 ip-10-0-131-202 kubenswrapper[2578]: E1030 21:52:48.910414    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wpdr9" podUID="8806f1e1-0057-4738-8fbc-7e503e1f91ca"
Oct 30 21:52:48.910695 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:48.910421    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4vtdc"
Oct 30 21:52:48.910695 ip-10-0-131-202 kubenswrapper[2578]: E1030 21:52:48.910471    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4vtdc" podUID="d17b7754-d1d4-4952-97c7-7a04c714b48c"
Oct 30 21:52:50.675021 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:50.674997    2578 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-131-202.us-west-1.compute.internal" event="NodeReady"
Oct 30 21:52:50.675358 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:50.675115    2578 kubelet_node_status.go:477] "Fast updating node status as it just became ready"
Oct 30 21:52:50.777214 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:50.777188    2578 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-131-202.us-west-1.compute.internal" event="NodeReady"
Oct 30 21:52:50.826184 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:50.826092    2578 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-8qxwh"]
Oct 30 21:52:50.829177 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:50.829160    2578 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-h2pjp"]
Oct 30 21:52:50.829303 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:50.829286    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-8qxwh"
Oct 30 21:52:50.831753 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:50.831731    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-h2pjp"
Oct 30 21:52:50.838555 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:50.838535    2578 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-h2pjp"]
Oct 30 21:52:50.841826 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:50.841769    2578 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-8qxwh"]
Oct 30 21:52:50.880729 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:50.880700    2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Oct 30 21:52:50.880838 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:50.880732    2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Oct 30 21:52:50.880902 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:50.880887    2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-gsd79\""
Oct 30 21:52:50.882829 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:50.882815    2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Oct 30 21:52:50.882998 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:50.882985    2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Oct 30 21:52:50.883206 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:50.883195    2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"ingress-canary-dockercfg-qt896\""
Oct 30 21:52:50.887026 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:50.887014    2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Oct 30 21:52:50.910303 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:50.910276    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-slfvl"
Oct 30 21:52:50.910303 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:50.910295    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wpdr9"
Oct 30 21:52:50.910439 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:50.910314    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4vtdc"
Oct 30 21:52:50.910439 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:50.910319    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dtsnb\" (UniqueName: \"kubernetes.io/projected/4a03a836-506f-404e-9c55-c26d779cd4ee-kube-api-access-dtsnb\") pod \"dns-default-8qxwh\" (UID: \"4a03a836-506f-404e-9c55-c26d779cd4ee\") " pod="openshift-dns/dns-default-8qxwh"
Oct 30 21:52:50.910439 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:50.910344    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2345ff5b-db97-4855-8adb-59b9a780615e-cert\") pod \"ingress-canary-h2pjp\" (UID: \"2345ff5b-db97-4855-8adb-59b9a780615e\") " pod="openshift-ingress-canary/ingress-canary-h2pjp"
Oct 30 21:52:50.910588 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:50.910436    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4a03a836-506f-404e-9c55-c26d779cd4ee-tmp-dir\") pod \"dns-default-8qxwh\" (UID: \"4a03a836-506f-404e-9c55-c26d779cd4ee\") " pod="openshift-dns/dns-default-8qxwh"
Oct 30 21:52:50.910588 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:50.910469    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c622t\" (UniqueName: \"kubernetes.io/projected/2345ff5b-db97-4855-8adb-59b9a780615e-kube-api-access-c622t\") pod \"ingress-canary-h2pjp\" (UID: \"2345ff5b-db97-4855-8adb-59b9a780615e\") " pod="openshift-ingress-canary/ingress-canary-h2pjp"
Oct 30 21:52:50.910588 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:50.910507    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/4a03a836-506f-404e-9c55-c26d779cd4ee-config-volume\") pod \"dns-default-8qxwh\" (UID: \"4a03a836-506f-404e-9c55-c26d779cd4ee\") " pod="openshift-dns/dns-default-8qxwh"
Oct 30 21:52:50.910724 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:50.910621    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4a03a836-506f-404e-9c55-c26d779cd4ee-metrics-tls\") pod \"dns-default-8qxwh\" (UID: \"4a03a836-506f-404e-9c55-c26d779cd4ee\") " pod="openshift-dns/dns-default-8qxwh"
Oct 30 21:52:50.985453 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:50.985433    2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Oct 30 21:52:50.985649 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:50.985621    2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Oct 30 21:52:50.985920 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:50.985898    2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Oct 30 21:52:50.986045 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:50.986030    2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-7bphz\""
Oct 30 21:52:50.986161 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:50.986062    2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"global-pull-secret-syncer-dockercfg-6q7ft\""
Oct 30 21:52:50.986161 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:50.986093    2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Oct 30 21:52:50.986309 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:50.986288    2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-check-target-dockercfg-gmfbd\""
Oct 30 21:52:51.011619 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:51.011596    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4a03a836-506f-404e-9c55-c26d779cd4ee-tmp-dir\") pod \"dns-default-8qxwh\" (UID: \"4a03a836-506f-404e-9c55-c26d779cd4ee\") " pod="openshift-dns/dns-default-8qxwh"
Oct 30 21:52:51.011723 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:51.011626    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c622t\" (UniqueName: \"kubernetes.io/projected/2345ff5b-db97-4855-8adb-59b9a780615e-kube-api-access-c622t\") pod \"ingress-canary-h2pjp\" (UID: \"2345ff5b-db97-4855-8adb-59b9a780615e\") " pod="openshift-ingress-canary/ingress-canary-h2pjp"
Oct 30 21:52:51.011723 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:51.011659    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/4a03a836-506f-404e-9c55-c26d779cd4ee-config-volume\") pod \"dns-default-8qxwh\" (UID: \"4a03a836-506f-404e-9c55-c26d779cd4ee\") " pod="openshift-dns/dns-default-8qxwh"
Oct 30 21:52:51.011723 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:51.011689    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4a03a836-506f-404e-9c55-c26d779cd4ee-metrics-tls\") pod \"dns-default-8qxwh\" (UID: \"4a03a836-506f-404e-9c55-c26d779cd4ee\") " pod="openshift-dns/dns-default-8qxwh"
Oct 30 21:52:51.011881 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:51.011749    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dtsnb\" (UniqueName: \"kubernetes.io/projected/4a03a836-506f-404e-9c55-c26d779cd4ee-kube-api-access-dtsnb\") pod \"dns-default-8qxwh\" (UID: \"4a03a836-506f-404e-9c55-c26d779cd4ee\") " pod="openshift-dns/dns-default-8qxwh"
Oct 30 21:52:51.011881 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:51.011772    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2345ff5b-db97-4855-8adb-59b9a780615e-cert\") pod \"ingress-canary-h2pjp\" (UID: \"2345ff5b-db97-4855-8adb-59b9a780615e\") " pod="openshift-ingress-canary/ingress-canary-h2pjp"
Oct 30 21:52:51.011881 ip-10-0-131-202 kubenswrapper[2578]: E1030 21:52:51.011851    2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Oct 30 21:52:51.012110 ip-10-0-131-202 kubenswrapper[2578]: E1030 21:52:51.011910    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4a03a836-506f-404e-9c55-c26d779cd4ee-metrics-tls podName:4a03a836-506f-404e-9c55-c26d779cd4ee nodeName:}" failed. No retries permitted until 2026-10-30 21:52:51.511891102 +0000 UTC m=+32.151591317 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4a03a836-506f-404e-9c55-c26d779cd4ee-metrics-tls") pod "dns-default-8qxwh" (UID: "4a03a836-506f-404e-9c55-c26d779cd4ee") : secret "dns-default-metrics-tls" not found
Oct 30 21:52:51.012110 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:51.011983    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4a03a836-506f-404e-9c55-c26d779cd4ee-tmp-dir\") pod \"dns-default-8qxwh\" (UID: \"4a03a836-506f-404e-9c55-c26d779cd4ee\") " pod="openshift-dns/dns-default-8qxwh"
Oct 30 21:52:51.012367 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:51.012348    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/4a03a836-506f-404e-9c55-c26d779cd4ee-config-volume\") pod \"dns-default-8qxwh\" (UID: \"4a03a836-506f-404e-9c55-c26d779cd4ee\") " pod="openshift-dns/dns-default-8qxwh"
Oct 30 21:52:51.015498 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:51.015478    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2345ff5b-db97-4855-8adb-59b9a780615e-cert\") pod \"ingress-canary-h2pjp\" (UID: \"2345ff5b-db97-4855-8adb-59b9a780615e\") " pod="openshift-ingress-canary/ingress-canary-h2pjp"
Oct 30 21:52:51.069582 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:51.069550    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c622t\" (UniqueName: \"kubernetes.io/projected/2345ff5b-db97-4855-8adb-59b9a780615e-kube-api-access-c622t\") pod \"ingress-canary-h2pjp\" (UID: \"2345ff5b-db97-4855-8adb-59b9a780615e\") " pod="openshift-ingress-canary/ingress-canary-h2pjp"
Oct 30 21:52:51.069930 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:51.069911    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dtsnb\" (UniqueName: \"kubernetes.io/projected/4a03a836-506f-404e-9c55-c26d779cd4ee-kube-api-access-dtsnb\") pod \"dns-default-8qxwh\" (UID: \"4a03a836-506f-404e-9c55-c26d779cd4ee\") " pod="openshift-dns/dns-default-8qxwh"
Oct 30 21:52:51.148396 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:51.148331    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-h2pjp"
Oct 30 21:52:51.515175 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:51.515138    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4a03a836-506f-404e-9c55-c26d779cd4ee-metrics-tls\") pod \"dns-default-8qxwh\" (UID: \"4a03a836-506f-404e-9c55-c26d779cd4ee\") " pod="openshift-dns/dns-default-8qxwh"
Oct 30 21:52:51.515334 ip-10-0-131-202 kubenswrapper[2578]: E1030 21:52:51.515234    2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Oct 30 21:52:51.515334 ip-10-0-131-202 kubenswrapper[2578]: E1030 21:52:51.515299    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4a03a836-506f-404e-9c55-c26d779cd4ee-metrics-tls podName:4a03a836-506f-404e-9c55-c26d779cd4ee nodeName:}" failed. No retries permitted until 2026-10-30 21:52:52.515283972 +0000 UTC m=+33.154984189 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4a03a836-506f-404e-9c55-c26d779cd4ee-metrics-tls") pod "dns-default-8qxwh" (UID: "4a03a836-506f-404e-9c55-c26d779cd4ee") : secret "dns-default-metrics-tls" not found
Oct 30 21:52:51.532825 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:51.532801    2578 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-h2pjp"]
Oct 30 21:52:51.536458 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:51.536435    2578 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2345ff5b_db97_4855_8adb_59b9a780615e.slice/crio-c8ab0219dae8058a1a0ad4e07bca8ce1cc695a209049fe6f3e048c5e56ae3da0 WatchSource:0}: Error finding container c8ab0219dae8058a1a0ad4e07bca8ce1cc695a209049fe6f3e048c5e56ae3da0: Status 404 returned error can't find the container with id c8ab0219dae8058a1a0ad4e07bca8ce1cc695a209049fe6f3e048c5e56ae3da0
Oct 30 21:52:52.139710 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:52.139672    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-h2pjp" event={"ID":"2345ff5b-db97-4855-8adb-59b9a780615e","Type":"ContainerStarted","Data":"c8ab0219dae8058a1a0ad4e07bca8ce1cc695a209049fe6f3e048c5e56ae3da0"}
Oct 30 21:52:52.521016 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:52.520972    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4a03a836-506f-404e-9c55-c26d779cd4ee-metrics-tls\") pod \"dns-default-8qxwh\" (UID: \"4a03a836-506f-404e-9c55-c26d779cd4ee\") " pod="openshift-dns/dns-default-8qxwh"
Oct 30 21:52:52.521179 ip-10-0-131-202 kubenswrapper[2578]: E1030 21:52:52.521141    2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Oct 30 21:52:52.521223 ip-10-0-131-202 kubenswrapper[2578]: E1030 21:52:52.521205    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4a03a836-506f-404e-9c55-c26d779cd4ee-metrics-tls podName:4a03a836-506f-404e-9c55-c26d779cd4ee nodeName:}" failed. No retries permitted until 2026-10-30 21:52:54.521190638 +0000 UTC m=+35.160890842 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4a03a836-506f-404e-9c55-c26d779cd4ee-metrics-tls") pod "dns-default-8qxwh" (UID: "4a03a836-506f-404e-9c55-c26d779cd4ee") : secret "dns-default-metrics-tls" not found
Oct 30 21:52:52.824207 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:52.824078    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8806f1e1-0057-4738-8fbc-7e503e1f91ca-metrics-certs\") pod \"network-metrics-daemon-wpdr9\" (UID: \"8806f1e1-0057-4738-8fbc-7e503e1f91ca\") " pod="openshift-multus/network-metrics-daemon-wpdr9"
Oct 30 21:52:52.824207 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:52.824176    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d17b7754-d1d4-4952-97c7-7a04c714b48c-original-pull-secret\") pod \"global-pull-secret-syncer-4vtdc\" (UID: \"d17b7754-d1d4-4952-97c7-7a04c714b48c\") " pod="kube-system/global-pull-secret-syncer-4vtdc"
Oct 30 21:52:52.824470 ip-10-0-131-202 kubenswrapper[2578]: E1030 21:52:52.824260    2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Oct 30 21:52:52.824470 ip-10-0-131-202 kubenswrapper[2578]: E1030 21:52:52.824339    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8806f1e1-0057-4738-8fbc-7e503e1f91ca-metrics-certs podName:8806f1e1-0057-4738-8fbc-7e503e1f91ca nodeName:}" failed. No retries permitted until 2026-10-30 21:53:24.82431973 +0000 UTC m=+65.464019935 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8806f1e1-0057-4738-8fbc-7e503e1f91ca-metrics-certs") pod "network-metrics-daemon-wpdr9" (UID: "8806f1e1-0057-4738-8fbc-7e503e1f91ca") : secret "metrics-daemon-secret" not found
Oct 30 21:52:52.826911 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:52.826885    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d17b7754-d1d4-4952-97c7-7a04c714b48c-original-pull-secret\") pod \"global-pull-secret-syncer-4vtdc\" (UID: \"d17b7754-d1d4-4952-97c7-7a04c714b48c\") " pod="kube-system/global-pull-secret-syncer-4vtdc"
Oct 30 21:52:52.924736 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:52.924705    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6j99v\" (UniqueName: \"kubernetes.io/projected/7be00c31-281f-4d1c-8e12-1bf74bbe7cbd-kube-api-access-6j99v\") pod \"network-check-target-slfvl\" (UID: \"7be00c31-281f-4d1c-8e12-1bf74bbe7cbd\") " pod="openshift-network-diagnostics/network-check-target-slfvl"
Oct 30 21:52:52.927783 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:52.927698    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6j99v\" (UniqueName: \"kubernetes.io/projected/7be00c31-281f-4d1c-8e12-1bf74bbe7cbd-kube-api-access-6j99v\") pod \"network-check-target-slfvl\" (UID: \"7be00c31-281f-4d1c-8e12-1bf74bbe7cbd\") " pod="openshift-network-diagnostics/network-check-target-slfvl"
Oct 30 21:52:53.021728 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:53.021698    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-slfvl"
Oct 30 21:52:53.028028 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:53.027973    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4vtdc"
Oct 30 21:52:53.420661 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:53.420578    2578 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-slfvl"]
Oct 30 21:52:53.425109 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:53.425077    2578 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7be00c31_281f_4d1c_8e12_1bf74bbe7cbd.slice/crio-eecea248f97ac1f5faa4ef08a1b96061cee2cce9cac0767037b2d544ff1a8c1d WatchSource:0}: Error finding container eecea248f97ac1f5faa4ef08a1b96061cee2cce9cac0767037b2d544ff1a8c1d: Status 404 returned error can't find the container with id eecea248f97ac1f5faa4ef08a1b96061cee2cce9cac0767037b2d544ff1a8c1d
Oct 30 21:52:53.470082 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:53.470049    2578 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-4vtdc"]
Oct 30 21:52:53.474063 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:53.474035    2578 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd17b7754_d1d4_4952_97c7_7a04c714b48c.slice/crio-3f3feb24d73c361bf3c8b6abc3c8ab7edc3aa0f9549ddbec02315f3dd7ed0b0c WatchSource:0}: Error finding container 3f3feb24d73c361bf3c8b6abc3c8ab7edc3aa0f9549ddbec02315f3dd7ed0b0c: Status 404 returned error can't find the container with id 3f3feb24d73c361bf3c8b6abc3c8ab7edc3aa0f9549ddbec02315f3dd7ed0b0c
Oct 30 21:52:54.146735 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:54.146696    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-4vtdc" event={"ID":"d17b7754-d1d4-4952-97c7-7a04c714b48c","Type":"ContainerStarted","Data":"3f3feb24d73c361bf3c8b6abc3c8ab7edc3aa0f9549ddbec02315f3dd7ed0b0c"}
Oct 30 21:52:54.147784 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:54.147758    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-slfvl" event={"ID":"7be00c31-281f-4d1c-8e12-1bf74bbe7cbd","Type":"ContainerStarted","Data":"eecea248f97ac1f5faa4ef08a1b96061cee2cce9cac0767037b2d544ff1a8c1d"}
Oct 30 21:52:54.536910 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:54.536869    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4a03a836-506f-404e-9c55-c26d779cd4ee-metrics-tls\") pod \"dns-default-8qxwh\" (UID: \"4a03a836-506f-404e-9c55-c26d779cd4ee\") " pod="openshift-dns/dns-default-8qxwh"
Oct 30 21:52:54.537410 ip-10-0-131-202 kubenswrapper[2578]: E1030 21:52:54.537050    2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Oct 30 21:52:54.537410 ip-10-0-131-202 kubenswrapper[2578]: E1030 21:52:54.537115    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4a03a836-506f-404e-9c55-c26d779cd4ee-metrics-tls podName:4a03a836-506f-404e-9c55-c26d779cd4ee nodeName:}" failed. No retries permitted until 2026-10-30 21:52:58.537098061 +0000 UTC m=+39.176798261 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4a03a836-506f-404e-9c55-c26d779cd4ee-metrics-tls") pod "dns-default-8qxwh" (UID: "4a03a836-506f-404e-9c55-c26d779cd4ee") : secret "dns-default-metrics-tls" not found
Oct 30 21:52:56.155079 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:56.155039    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-h2pjp" event={"ID":"2345ff5b-db97-4855-8adb-59b9a780615e","Type":"ContainerStarted","Data":"801e796fff9ac6959c2e3cc9716c1091080aad11c0592e90dac3706e322a31d2"}
Oct 30 21:52:56.273977 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:56.273918    2578 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-h2pjp" podStartSLOduration=2.6945171759999997 podStartE2EDuration="6.273896292s" totalImagesPullingTime="3.579379116s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:52:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:52:56.27315742 +0000 UTC m=+36.912857633" watchObservedRunningTime="2026-10-30 21:52:56.273896292 +0000 UTC m=+36.913596517"
Oct 30 21:52:57.883072 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:57.882984    2578 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-bkhmz"]
Oct 30 21:52:57.898532 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:57.898502    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-bkhmz"
Oct 30 21:52:58.002685 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:58.002639    2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Oct 30 21:52:58.002834 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:58.002807    2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-h6mh8\""
Oct 30 21:52:58.003687 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:58.003665    2578 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-bkhmz"]
Oct 30 21:52:58.039356 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:58.039311    2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Oct 30 21:52:58.039356 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:58.039329    2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Oct 30 21:52:58.039356 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:58.039321    2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Oct 30 21:52:58.066744 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:58.066714    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/db073c3b-7cee-4e22-8c11-209bbeb21a96-crio-socket\") pod \"insights-runtime-extractor-bkhmz\" (UID: \"db073c3b-7cee-4e22-8c11-209bbeb21a96\") " pod="openshift-insights/insights-runtime-extractor-bkhmz"
Oct 30 21:52:58.066905 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:58.066788    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/db073c3b-7cee-4e22-8c11-209bbeb21a96-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-bkhmz\" (UID: \"db073c3b-7cee-4e22-8c11-209bbeb21a96\") " pod="openshift-insights/insights-runtime-extractor-bkhmz"
Oct 30 21:52:58.066905 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:58.066816    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vd7d5\" (UniqueName: \"kubernetes.io/projected/db073c3b-7cee-4e22-8c11-209bbeb21a96-kube-api-access-vd7d5\") pod \"insights-runtime-extractor-bkhmz\" (UID: \"db073c3b-7cee-4e22-8c11-209bbeb21a96\") " pod="openshift-insights/insights-runtime-extractor-bkhmz"
Oct 30 21:52:58.066905 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:58.066845    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/db073c3b-7cee-4e22-8c11-209bbeb21a96-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-bkhmz\" (UID: \"db073c3b-7cee-4e22-8c11-209bbeb21a96\") " pod="openshift-insights/insights-runtime-extractor-bkhmz"
Oct 30 21:52:58.066905 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:58.066868    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/db073c3b-7cee-4e22-8c11-209bbeb21a96-data-volume\") pod \"insights-runtime-extractor-bkhmz\" (UID: \"db073c3b-7cee-4e22-8c11-209bbeb21a96\") " pod="openshift-insights/insights-runtime-extractor-bkhmz"
Oct 30 21:52:58.085157 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:58.085112    2578 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-66df5d496b-zbfpb"]
Oct 30 21:52:58.096633 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:58.096609    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-zbfpb"
Oct 30 21:52:58.145493 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:58.145381    2578 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-66df5d496b-zbfpb"]
Oct 30 21:52:58.150204 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:58.150112    2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\""
Oct 30 21:52:58.150204 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:58.150156    2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\""
Oct 30 21:52:58.150397 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:58.150272    2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-jfz4q\""
Oct 30 21:52:58.162755 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:58.162305    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-slfvl" event={"ID":"7be00c31-281f-4d1c-8e12-1bf74bbe7cbd","Type":"ContainerStarted","Data":"d41be9bb646808c3176f74a5cfbf39be9a646062bd3932cdda8de3b8075639cc"}
Oct 30 21:52:58.162755 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:58.162469    2578 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-slfvl"
Oct 30 21:52:58.167944 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:58.167906    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/db073c3b-7cee-4e22-8c11-209bbeb21a96-crio-socket\") pod \"insights-runtime-extractor-bkhmz\" (UID: \"db073c3b-7cee-4e22-8c11-209bbeb21a96\") " pod="openshift-insights/insights-runtime-extractor-bkhmz"
Oct 30 21:52:58.168077 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:58.167978    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/db073c3b-7cee-4e22-8c11-209bbeb21a96-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-bkhmz\" (UID: \"db073c3b-7cee-4e22-8c11-209bbeb21a96\") " pod="openshift-insights/insights-runtime-extractor-bkhmz"
Oct 30 21:52:58.168077 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:58.168006    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vd7d5\" (UniqueName: \"kubernetes.io/projected/db073c3b-7cee-4e22-8c11-209bbeb21a96-kube-api-access-vd7d5\") pod \"insights-runtime-extractor-bkhmz\" (UID: \"db073c3b-7cee-4e22-8c11-209bbeb21a96\") " pod="openshift-insights/insights-runtime-extractor-bkhmz"
Oct 30 21:52:58.168077 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:58.168028    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/db073c3b-7cee-4e22-8c11-209bbeb21a96-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-bkhmz\" (UID: \"db073c3b-7cee-4e22-8c11-209bbeb21a96\") " pod="openshift-insights/insights-runtime-extractor-bkhmz"
Oct 30 21:52:58.168077 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:58.168052    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fsf4l\" (UniqueName: \"kubernetes.io/projected/7e6b0df0-07eb-4301-adaa-1a6514639d9c-kube-api-access-fsf4l\") pod \"migrator-66df5d496b-zbfpb\" (UID: \"7e6b0df0-07eb-4301-adaa-1a6514639d9c\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-zbfpb"
Oct 30 21:52:58.168301 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:58.168078    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/db073c3b-7cee-4e22-8c11-209bbeb21a96-data-volume\") pod \"insights-runtime-extractor-bkhmz\" (UID: \"db073c3b-7cee-4e22-8c11-209bbeb21a96\") " pod="openshift-insights/insights-runtime-extractor-bkhmz"
Oct 30 21:52:58.168301 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:58.168161    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/db073c3b-7cee-4e22-8c11-209bbeb21a96-crio-socket\") pod \"insights-runtime-extractor-bkhmz\" (UID: \"db073c3b-7cee-4e22-8c11-209bbeb21a96\") " pod="openshift-insights/insights-runtime-extractor-bkhmz"
Oct 30 21:52:58.168301 ip-10-0-131-202 kubenswrapper[2578]: E1030 21:52:58.168165    2578 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Oct 30 21:52:58.168301 ip-10-0-131-202 kubenswrapper[2578]: E1030 21:52:58.168239    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/db073c3b-7cee-4e22-8c11-209bbeb21a96-insights-runtime-extractor-tls podName:db073c3b-7cee-4e22-8c11-209bbeb21a96 nodeName:}" failed. No retries permitted until 2026-10-30 21:52:58.668220529 +0000 UTC m=+39.307920743 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/db073c3b-7cee-4e22-8c11-209bbeb21a96-insights-runtime-extractor-tls") pod "insights-runtime-extractor-bkhmz" (UID: "db073c3b-7cee-4e22-8c11-209bbeb21a96") : secret "insights-runtime-extractor-tls" not found
Oct 30 21:52:58.168301 ip-10-0-131-202 kubenswrapper[2578]: E1030 21:52:58.168259    2578 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Oct 30 21:52:58.168533 ip-10-0-131-202 kubenswrapper[2578]: E1030 21:52:58.168319    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/db073c3b-7cee-4e22-8c11-209bbeb21a96-kube-rbac-proxy-cm podName:db073c3b-7cee-4e22-8c11-209bbeb21a96 nodeName:}" failed. No retries permitted until 2026-10-30 21:52:58.668303755 +0000 UTC m=+39.308003984 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/db073c3b-7cee-4e22-8c11-209bbeb21a96-kube-rbac-proxy-cm") pod "insights-runtime-extractor-bkhmz" (UID: "db073c3b-7cee-4e22-8c11-209bbeb21a96") : configmap "kube-rbac-proxy" not found
Oct 30 21:52:58.168582 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:58.168567    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/db073c3b-7cee-4e22-8c11-209bbeb21a96-data-volume\") pod \"insights-runtime-extractor-bkhmz\" (UID: \"db073c3b-7cee-4e22-8c11-209bbeb21a96\") " pod="openshift-insights/insights-runtime-extractor-bkhmz"
Oct 30 21:52:58.230510 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:58.230476    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vd7d5\" (UniqueName: \"kubernetes.io/projected/db073c3b-7cee-4e22-8c11-209bbeb21a96-kube-api-access-vd7d5\") pod \"insights-runtime-extractor-bkhmz\" (UID: \"db073c3b-7cee-4e22-8c11-209bbeb21a96\") " pod="openshift-insights/insights-runtime-extractor-bkhmz"
Oct 30 21:52:58.268693 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:58.268665    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fsf4l\" (UniqueName: \"kubernetes.io/projected/7e6b0df0-07eb-4301-adaa-1a6514639d9c-kube-api-access-fsf4l\") pod \"migrator-66df5d496b-zbfpb\" (UID: \"7e6b0df0-07eb-4301-adaa-1a6514639d9c\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-zbfpb"
Oct 30 21:52:58.322815 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:58.322750    2578 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-slfvl" podStartSLOduration=34.255014848 podStartE2EDuration="38.322730574s" totalImagesPullingTime="4.067715726s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:52:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:52:58.321622056 +0000 UTC m=+38.961322279" watchObservedRunningTime="2026-10-30 21:52:58.322730574 +0000 UTC m=+38.962430797"
Oct 30 21:52:58.326847 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:58.326818    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fsf4l\" (UniqueName: \"kubernetes.io/projected/7e6b0df0-07eb-4301-adaa-1a6514639d9c-kube-api-access-fsf4l\") pod \"migrator-66df5d496b-zbfpb\" (UID: \"7e6b0df0-07eb-4301-adaa-1a6514639d9c\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-zbfpb"
Oct 30 21:52:58.416019 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:58.415948    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-zbfpb"
Oct 30 21:52:58.572176 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:58.572135    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4a03a836-506f-404e-9c55-c26d779cd4ee-metrics-tls\") pod \"dns-default-8qxwh\" (UID: \"4a03a836-506f-404e-9c55-c26d779cd4ee\") " pod="openshift-dns/dns-default-8qxwh"
Oct 30 21:52:58.572344 ip-10-0-131-202 kubenswrapper[2578]: E1030 21:52:58.572264    2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Oct 30 21:52:58.572415 ip-10-0-131-202 kubenswrapper[2578]: E1030 21:52:58.572358    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4a03a836-506f-404e-9c55-c26d779cd4ee-metrics-tls podName:4a03a836-506f-404e-9c55-c26d779cd4ee nodeName:}" failed. No retries permitted until 2026-10-30 21:53:06.572337431 +0000 UTC m=+47.212037634 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4a03a836-506f-404e-9c55-c26d779cd4ee-metrics-tls") pod "dns-default-8qxwh" (UID: "4a03a836-506f-404e-9c55-c26d779cd4ee") : secret "dns-default-metrics-tls" not found
Oct 30 21:52:58.672778 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:58.672703    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/db073c3b-7cee-4e22-8c11-209bbeb21a96-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-bkhmz\" (UID: \"db073c3b-7cee-4e22-8c11-209bbeb21a96\") " pod="openshift-insights/insights-runtime-extractor-bkhmz"
Oct 30 21:52:58.672778 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:58.672752    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/db073c3b-7cee-4e22-8c11-209bbeb21a96-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-bkhmz\" (UID: \"db073c3b-7cee-4e22-8c11-209bbeb21a96\") " pod="openshift-insights/insights-runtime-extractor-bkhmz"
Oct 30 21:52:58.672978 ip-10-0-131-202 kubenswrapper[2578]: E1030 21:52:58.672871    2578 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Oct 30 21:52:58.672978 ip-10-0-131-202 kubenswrapper[2578]: E1030 21:52:58.672943    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/db073c3b-7cee-4e22-8c11-209bbeb21a96-insights-runtime-extractor-tls podName:db073c3b-7cee-4e22-8c11-209bbeb21a96 nodeName:}" failed. No retries permitted until 2026-10-30 21:52:59.672920913 +0000 UTC m=+40.312621114 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/db073c3b-7cee-4e22-8c11-209bbeb21a96-insights-runtime-extractor-tls") pod "insights-runtime-extractor-bkhmz" (UID: "db073c3b-7cee-4e22-8c11-209bbeb21a96") : secret "insights-runtime-extractor-tls" not found
Oct 30 21:52:58.672978 ip-10-0-131-202 kubenswrapper[2578]: E1030 21:52:58.672875    2578 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Oct 30 21:52:58.673154 ip-10-0-131-202 kubenswrapper[2578]: E1030 21:52:58.673007    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/db073c3b-7cee-4e22-8c11-209bbeb21a96-kube-rbac-proxy-cm podName:db073c3b-7cee-4e22-8c11-209bbeb21a96 nodeName:}" failed. No retries permitted until 2026-10-30 21:52:59.672992139 +0000 UTC m=+40.312692343 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/db073c3b-7cee-4e22-8c11-209bbeb21a96-kube-rbac-proxy-cm") pod "insights-runtime-extractor-bkhmz" (UID: "db073c3b-7cee-4e22-8c11-209bbeb21a96") : configmap "kube-rbac-proxy" not found
Oct 30 21:52:58.805042 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:58.804715    2578 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-66df5d496b-zbfpb"]
Oct 30 21:52:58.816076 ip-10-0-131-202 kubenswrapper[2578]: W1030 21:52:58.816048    2578 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7e6b0df0_07eb_4301_adaa_1a6514639d9c.slice/crio-5a0d9b46d476286e5b9f13aa9ed77125183273c6d32c4e1fb87b7608b8e29b35 WatchSource:0}: Error finding container 5a0d9b46d476286e5b9f13aa9ed77125183273c6d32c4e1fb87b7608b8e29b35: Status 404 returned error can't find the container with id 5a0d9b46d476286e5b9f13aa9ed77125183273c6d32c4e1fb87b7608b8e29b35
Oct 30 21:52:59.165872 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:59.165844    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-zbfpb" event={"ID":"7e6b0df0-07eb-4301-adaa-1a6514639d9c","Type":"ContainerStarted","Data":"5a0d9b46d476286e5b9f13aa9ed77125183273c6d32c4e1fb87b7608b8e29b35"}
Oct 30 21:52:59.433318 ip-10-0-131-202 kubenswrapper[2578]: I1030 21:52:59.433222    2578 dynamic_cafile_content.go:175] "Shutting down controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Oct 30 21:52:59.433371 ip-10-0-131-202 systemd[1]: Stopping Kubernetes Kubelet...
Oct 30 21:52:59.444503 ip-10-0-131-202 systemd[1]: kubelet.service: Deactivated successfully.
Oct 30 21:52:59.444723 ip-10-0-131-202 systemd[1]: Stopped Kubernetes Kubelet.
Oct 30 21:52:59.444782 ip-10-0-131-202 systemd[1]: kubelet.service: Consumed 2.191s CPU time.
Oct 30 21:52:59.453758 ip-10-0-131-202 systemd[1]: Starting Kubernetes Kubelet...
Oct 30 21:52:59.541416 ip-10-0-131-202 kubenswrapper[4412]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 30 21:52:59.541416 ip-10-0-131-202 kubenswrapper[4412]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Oct 30 21:52:59.541416 ip-10-0-131-202 kubenswrapper[4412]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 30 21:52:59.541416 ip-10-0-131-202 kubenswrapper[4412]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 30 21:52:59.544700 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.544621    4412 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 21:52:59.544700 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.544698    4412 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 21:52:59.544700 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.544702    4412 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 21:52:59.544700 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.544707    4412 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 21:52:59.544881 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.544713    4412 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 21:52:59.544881 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.544716    4412 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 21:52:59.544881 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.544719    4412 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 21:52:59.544881 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.544723    4412 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 21:52:59.544881 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.544726    4412 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 21:52:59.544881 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.544729    4412 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 21:52:59.544881 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.544731    4412 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 21:52:59.544881 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.544734    4412 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 21:52:59.544881 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.544736    4412 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 21:52:59.544881 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.544739    4412 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 21:52:59.544881 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.544742    4412 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 21:52:59.544881 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.544744    4412 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 21:52:59.544881 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.544747    4412 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 21:52:59.544881 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.544750    4412 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 21:52:59.544881 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.544752    4412 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 21:52:59.544881 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.544756    4412 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 21:52:59.544881 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.544760    4412 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 21:52:59.544881 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.544763    4412 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 21:52:59.544881 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.544766    4412 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 21:52:59.545424 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.544768    4412 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 21:52:59.545424 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.544771    4412 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 21:52:59.545424 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.544773    4412 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 21:52:59.545424 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.544776    4412 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 21:52:59.545424 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.544778    4412 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 21:52:59.545424 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.544781    4412 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 21:52:59.545424 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.544784    4412 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 21:52:59.545424 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.544786    4412 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 21:52:59.545424 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.544789    4412 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 21:52:59.545424 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.544792    4412 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 21:52:59.545424 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.544794    4412 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 21:52:59.545424 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.544797    4412 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 21:52:59.545424 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.544800    4412 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 21:52:59.545424 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.544802    4412 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 21:52:59.545424 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.544805    4412 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 21:52:59.545424 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.544807    4412 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 21:52:59.545424 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.544810    4412 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 21:52:59.545424 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.544812    4412 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 21:52:59.545424 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.544815    4412 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 21:52:59.545424 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.544817    4412 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 21:52:59.545918 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.544820    4412 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 21:52:59.545918 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.544823    4412 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 21:52:59.545918 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.544825    4412 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 21:52:59.545918 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.544828    4412 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 21:52:59.545918 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.544830    4412 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 21:52:59.545918 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.544832    4412 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 21:52:59.545918 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.544835    4412 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 21:52:59.545918 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.544837    4412 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 21:52:59.545918 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.544840    4412 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 21:52:59.545918 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.544842    4412 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 21:52:59.545918 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.544845    4412 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 21:52:59.545918 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.544847    4412 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 21:52:59.545918 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.544850    4412 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 21:52:59.545918 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.544852    4412 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 21:52:59.545918 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.544856    4412 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 21:52:59.545918 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.544858    4412 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 21:52:59.545918 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.544861    4412 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 21:52:59.545918 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.544863    4412 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 21:52:59.545918 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.544866    4412 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 21:52:59.546402 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.544869    4412 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 21:52:59.546402 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.544872    4412 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 21:52:59.546402 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.544874    4412 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 21:52:59.546402 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.544877    4412 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 21:52:59.546402 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.544879    4412 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 21:52:59.546402 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.544882    4412 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 21:52:59.546402 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.544885    4412 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 21:52:59.546402 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.544888    4412 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 21:52:59.546402 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.544890    4412 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 21:52:59.546402 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.544893    4412 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 21:52:59.546402 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.544895    4412 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 21:52:59.546402 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.544898    4412 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 21:52:59.546402 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.544900    4412 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 21:52:59.546402 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.544903    4412 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 21:52:59.546402 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.544906    4412 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 21:52:59.546402 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.544908    4412 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 21:52:59.546402 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.544910    4412 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 21:52:59.546402 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.544913    4412 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 21:52:59.546402 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.544915    4412 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 21:52:59.546402 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.544918    4412 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 21:52:59.546910 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.544920    4412 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 21:52:59.546910 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.544924    4412 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 21:52:59.546910 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.544926    4412 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 21:52:59.546910 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.544929    4412 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 21:52:59.546910 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.544931    4412 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 21:52:59.546910 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.544934    4412 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 21:52:59.546910 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.544937    4412 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 21:52:59.546910 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.544939    4412 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 21:52:59.546910 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.544942    4412 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 21:52:59.546910 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.544945    4412 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 21:52:59.546910 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.544947    4412 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 21:52:59.546910 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.544950    4412 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 21:52:59.546910 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.544952    4412 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 21:52:59.546910 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.544955    4412 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 21:52:59.546910 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.544957    4412 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 21:52:59.546910 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.544960    4412 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 21:52:59.546910 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.544962    4412 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 21:52:59.546910 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.544964    4412 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 21:52:59.546910 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.544967    4412 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 21:52:59.546910 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.544969    4412 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 21:52:59.547455 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.544972    4412 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 21:52:59.547455 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.544974    4412 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 21:52:59.547455 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.544976    4412 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 21:52:59.547455 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.544979    4412 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 21:52:59.547455 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.545723    4412 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 21:52:59.547455 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.545730    4412 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 21:52:59.547455 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.545733    4412 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 21:52:59.547455 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.545735    4412 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 21:52:59.547455 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.545738    4412 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 21:52:59.547455 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.545741    4412 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 21:52:59.547455 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.545744    4412 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 21:52:59.547455 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.545747    4412 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 21:52:59.547455 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.545749    4412 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 21:52:59.547455 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.545752    4412 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 21:52:59.547455 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.545756    4412 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 21:52:59.547455 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.545759    4412 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 21:52:59.547455 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.545762    4412 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 21:52:59.547455 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.545765    4412 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 21:52:59.547455 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.545767    4412 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 21:52:59.547920 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.545770    4412 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 21:52:59.547920 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.545772    4412 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 21:52:59.547920 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.545775    4412 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 21:52:59.547920 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.545777    4412 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 21:52:59.547920 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.545780    4412 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 21:52:59.547920 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.545782    4412 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 21:52:59.547920 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.545785    4412 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 21:52:59.547920 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.545788    4412 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 21:52:59.547920 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.545790    4412 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 21:52:59.547920 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.545793    4412 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 21:52:59.547920 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.545796    4412 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 21:52:59.547920 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.545798    4412 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 21:52:59.547920 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.545801    4412 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 21:52:59.547920 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.545803    4412 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 21:52:59.547920 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.545806    4412 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 21:52:59.547920 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.545808    4412 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 21:52:59.547920 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.545811    4412 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 21:52:59.547920 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.545813    4412 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 21:52:59.547920 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.545816    4412 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 21:52:59.547920 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.545818    4412 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 21:52:59.548427 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.545821    4412 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 21:52:59.548427 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.545823    4412 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 21:52:59.548427 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.545826    4412 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 21:52:59.548427 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.545829    4412 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 21:52:59.548427 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.545831    4412 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 21:52:59.548427 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.545834    4412 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 21:52:59.548427 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.545839    4412 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 21:52:59.548427 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.545842    4412 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 21:52:59.548427 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.545846    4412 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 21:52:59.548427 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.545848    4412 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 21:52:59.548427 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.545851    4412 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 21:52:59.548427 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.545853    4412 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 21:52:59.548427 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.545856    4412 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 21:52:59.548427 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.545858    4412 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 21:52:59.548427 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.545861    4412 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 21:52:59.548427 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.545863    4412 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 21:52:59.548427 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.545866    4412 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 21:52:59.548427 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.545868    4412 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 21:52:59.548427 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.545873    4412 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 21:52:59.548930 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.545876    4412 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 21:52:59.548930 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.545879    4412 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 21:52:59.548930 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.545882    4412 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 21:52:59.548930 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.545884    4412 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 21:52:59.548930 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.545887    4412 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 21:52:59.548930 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.545890    4412 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 21:52:59.548930 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.545892    4412 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 21:52:59.548930 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.545895    4412 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 21:52:59.548930 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.545898    4412 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 21:52:59.548930 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.545900    4412 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 21:52:59.548930 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.545903    4412 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 21:52:59.548930 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.545905    4412 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 21:52:59.548930 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.545908    4412 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 21:52:59.548930 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.545911    4412 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 21:52:59.548930 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.545914    4412 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 21:52:59.548930 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.545916    4412 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 21:52:59.548930 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.545919    4412 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 21:52:59.548930 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.545921    4412 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 21:52:59.548930 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.545924    4412 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 21:52:59.548930 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.545930    4412 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 21:52:59.549462 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.545933    4412 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 21:52:59.549462 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.545936    4412 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 21:52:59.549462 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.545939    4412 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 21:52:59.549462 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.545941    4412 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 21:52:59.549462 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.545944    4412 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 21:52:59.549462 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.545946    4412 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 21:52:59.549462 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.545949    4412 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 21:52:59.549462 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.545951    4412 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 21:52:59.549462 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.545954    4412 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 21:52:59.549462 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.545957    4412 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 21:52:59.549462 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.545961    4412 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 21:52:59.549462 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.545964    4412 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 21:52:59.549462 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.545967    4412 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 21:52:59.549462 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.545970    4412 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 21:52:59.549462 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.545972    4412 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 21:52:59.549462 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.545975    4412 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 21:52:59.549462 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.545977    4412 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 21:52:59.549462 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.545980    4412 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 21:52:59.549462 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.545982    4412 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 21:52:59.562617 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.545985    4412 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 21:52:59.562617 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.545987    4412 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 21:52:59.562617 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.545990    4412 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 21:52:59.562617 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.545994    4412 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 21:52:59.562617 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.545997    4412 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 21:52:59.562617 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.546000    4412 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 21:52:59.562617 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.546002    4412 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 21:52:59.562617 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.546005    4412 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 21:52:59.562617 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.546007    4412 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 21:52:59.562617 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.546010    4412 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 21:52:59.562617 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.546012    4412 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 21:52:59.562617 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.546015    4412 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 21:52:59.562617 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.546018    4412 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 21:52:59.562617 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.546517    4412 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 21:52:59.562617 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.546523    4412 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 21:52:59.562617 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.546526    4412 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 21:52:59.562617 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.546529    4412 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 21:52:59.562617 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.546532    4412 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 21:52:59.562617 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.546534    4412 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 21:52:59.562617 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.546537    4412 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 21:52:59.563168 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.546540    4412 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 21:52:59.563168 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.546543    4412 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 21:52:59.563168 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.546545    4412 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 21:52:59.563168 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.546548    4412 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 21:52:59.563168 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.546550    4412 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 21:52:59.563168 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.546553    4412 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 21:52:59.563168 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.546555    4412 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 21:52:59.563168 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.546558    4412 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 21:52:59.563168 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.546561    4412 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 21:52:59.563168 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.546564    4412 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 21:52:59.563168 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.546567    4412 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 21:52:59.563168 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.546569    4412 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 21:52:59.563168 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.546572    4412 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 21:52:59.563168 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.546574    4412 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 21:52:59.563168 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.546577    4412 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 21:52:59.563168 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.546579    4412 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 21:52:59.563168 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.546582    4412 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 21:52:59.563168 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.546584    4412 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 21:52:59.563168 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.546587    4412 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 21:52:59.563633 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.546589    4412 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 21:52:59.563633 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.546592    4412 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 21:52:59.563633 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.546594    4412 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 21:52:59.563633 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.546596    4412 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 21:52:59.563633 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.546599    4412 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 21:52:59.563633 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.546603    4412 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 21:52:59.563633 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.546606    4412 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 21:52:59.563633 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.546609    4412 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 21:52:59.563633 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.546613    4412 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 21:52:59.563633 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.546619    4412 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 21:52:59.563633 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.546622    4412 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 21:52:59.563633 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.546625    4412 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 21:52:59.563633 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.546628    4412 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 21:52:59.563633 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.546631    4412 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 21:52:59.563633 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.546634    4412 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 21:52:59.563633 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.546637    4412 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 21:52:59.563633 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.546639    4412 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 21:52:59.563633 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.546642    4412 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 21:52:59.563633 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.546645    4412 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 21:52:59.564104 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.546647    4412 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 21:52:59.564104 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.546650    4412 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 21:52:59.564104 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.546653    4412 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 21:52:59.564104 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.546655    4412 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 21:52:59.564104 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.546657    4412 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 21:52:59.564104 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.546660    4412 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 21:52:59.564104 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.546662    4412 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 21:52:59.564104 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.546665    4412 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 21:52:59.564104 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.546668    4412 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 21:52:59.564104 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.546670    4412 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 21:52:59.564104 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.546672    4412 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 21:52:59.564104 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.546675    4412 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 21:52:59.564104 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.546677    4412 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 21:52:59.564104 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.546680    4412 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 21:52:59.564104 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.546682    4412 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 21:52:59.564104 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.546685    4412 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 21:52:59.564104 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.546687    4412 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 21:52:59.564104 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.546690    4412 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 21:52:59.564104 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.546694    4412 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 21:52:59.564104 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.546697    4412 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 21:52:59.564689 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.546699    4412 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 21:52:59.564689 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.546702    4412 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 21:52:59.564689 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.546705    4412 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 21:52:59.564689 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.546708    4412 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 21:52:59.564689 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.546711    4412 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 21:52:59.564689 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.546713    4412 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 21:52:59.564689 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.546716    4412 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 21:52:59.564689 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.546718    4412 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 21:52:59.564689 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.546721    4412 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 21:52:59.564689 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.546723    4412 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 21:52:59.564689 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.546726    4412 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 21:52:59.564689 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.546728    4412 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 21:52:59.564689 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.546730    4412 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 21:52:59.564689 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.546733    4412 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 21:52:59.564689 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.546735    4412 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 21:52:59.564689 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.546738    4412 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 21:52:59.564689 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.546740    4412 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 21:52:59.564689 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.546743    4412 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 21:52:59.564689 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.546745    4412 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 21:52:59.564689 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.546747    4412 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 21:52:59.565198 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.546750    4412 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 21:52:59.565198 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.546753    4412 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 21:52:59.565198 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.546755    4412 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 21:52:59.565198 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.546758    4412 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 21:52:59.565198 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.546760    4412 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 21:52:59.565198 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.546763    4412 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 21:52:59.565198 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.546765    4412 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 21:52:59.565198 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.546769    4412 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 21:52:59.565198 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.546772    4412 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 21:52:59.565198 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.546774    4412 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 21:52:59.565198 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.546779    4412 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 21:52:59.565198 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.546782    4412 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 21:52:59.565198 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.546784    4412 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 21:52:59.565198 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.546787    4412 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 21:52:59.565198 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.546789    4412 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 21:52:59.565198 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.546792    4412 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 21:52:59.565198 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.546795    4412 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 21:52:59.565198 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.546797    4412 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 21:52:59.565198 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.546800    4412 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 21:52:59.565669 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.546802    4412 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 21:52:59.565669 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.546806    4412 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 21:52:59.565669 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.546897    4412 flags.go:64] FLAG: --address="0.0.0.0"
Oct 30 21:52:59.565669 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.546907    4412 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Oct 30 21:52:59.565669 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.546924    4412 flags.go:64] FLAG: --anonymous-auth="true"
Oct 30 21:52:59.565669 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.546935    4412 flags.go:64] FLAG: --application-metrics-count-limit="100"
Oct 30 21:52:59.565669 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.546939    4412 flags.go:64] FLAG: --authentication-token-webhook="false"
Oct 30 21:52:59.565669 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.546942    4412 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Oct 30 21:52:59.565669 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.546947    4412 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Oct 30 21:52:59.565669 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.546950    4412 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Oct 30 21:52:59.565669 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.546954    4412 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Oct 30 21:52:59.565669 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.546956    4412 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Oct 30 21:52:59.565669 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.546960    4412 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Oct 30 21:52:59.565669 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.546963    4412 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Oct 30 21:52:59.565669 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.546966    4412 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Oct 30 21:52:59.565669 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.546970    4412 flags.go:64] FLAG: --cgroup-root=""
Oct 30 21:52:59.565669 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.546972    4412 flags.go:64] FLAG: --cgroups-per-qos="true"
Oct 30 21:52:59.565669 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.546975    4412 flags.go:64] FLAG: --client-ca-file=""
Oct 30 21:52:59.565669 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.546978    4412 flags.go:64] FLAG: --cloud-provider="external"
Oct 30 21:52:59.565669 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.546981    4412 flags.go:64] FLAG: --cluster-dns="[]"
Oct 30 21:52:59.565669 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.546985    4412 flags.go:64] FLAG: --cluster-domain=""
Oct 30 21:52:59.565669 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.546988    4412 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Oct 30 21:52:59.565669 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.546991    4412 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Oct 30 21:52:59.566252 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.546996    4412 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Oct 30 21:52:59.566252 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.546999    4412 flags.go:64] FLAG: --container-log-max-files="5"
Oct 30 21:52:59.566252 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.547003    4412 flags.go:64] FLAG: --container-log-max-size="10Mi"
Oct 30 21:52:59.566252 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.547006    4412 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Oct 30 21:52:59.566252 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.547009    4412 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Oct 30 21:52:59.566252 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.547012    4412 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Oct 30 21:52:59.566252 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.547015    4412 flags.go:64] FLAG: --contention-profiling="false"
Oct 30 21:52:59.566252 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.547018    4412 flags.go:64] FLAG: --cpu-cfs-quota="true"
Oct 30 21:52:59.566252 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.547021    4412 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Oct 30 21:52:59.566252 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.547024    4412 flags.go:64] FLAG: --cpu-manager-policy="none"
Oct 30 21:52:59.566252 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.547027    4412 flags.go:64] FLAG: --cpu-manager-policy-options=""
Oct 30 21:52:59.566252 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.547031    4412 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Oct 30 21:52:59.566252 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.547034    4412 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Oct 30 21:52:59.566252 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.547037    4412 flags.go:64] FLAG: --enable-debugging-handlers="true"
Oct 30 21:52:59.566252 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.547040    4412 flags.go:64] FLAG: --enable-load-reader="false"
Oct 30 21:52:59.566252 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.547043    4412 flags.go:64] FLAG: --enable-server="true"
Oct 30 21:52:59.566252 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.547045    4412 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Oct 30 21:52:59.566252 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.547049    4412 flags.go:64] FLAG: --event-burst="100"
Oct 30 21:52:59.566252 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.547052    4412 flags.go:64] FLAG: --event-qps="50"
Oct 30 21:52:59.566252 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.547055    4412 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Oct 30 21:52:59.566252 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.547058    4412 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Oct 30 21:52:59.566252 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.547061    4412 flags.go:64] FLAG: --eviction-hard=""
Oct 30 21:52:59.566252 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.547065    4412 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Oct 30 21:52:59.566252 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.547068    4412 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Oct 30 21:52:59.566833 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.547071    4412 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Oct 30 21:52:59.566833 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.547075    4412 flags.go:64] FLAG: --eviction-soft=""
Oct 30 21:52:59.566833 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.547078    4412 flags.go:64] FLAG: --eviction-soft-grace-period=""
Oct 30 21:52:59.566833 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.547080    4412 flags.go:64] FLAG: --exit-on-lock-contention="false"
Oct 30 21:52:59.566833 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.547083    4412 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Oct 30 21:52:59.566833 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.547086    4412 flags.go:64] FLAG: --experimental-mounter-path=""
Oct 30 21:52:59.566833 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.547089    4412 flags.go:64] FLAG: --fail-cgroupv1="true"
Oct 30 21:52:59.566833 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.547092    4412 flags.go:64] FLAG: --fail-swap-on="true"
Oct 30 21:52:59.566833 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.547096    4412 flags.go:64] FLAG: --feature-gates=""
Oct 30 21:52:59.566833 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.547100    4412 flags.go:64] FLAG: --file-check-frequency="20s"
Oct 30 21:52:59.566833 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.547103    4412 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Oct 30 21:52:59.566833 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.547106    4412 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Oct 30 21:52:59.566833 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.547109    4412 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Oct 30 21:52:59.566833 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.547112    4412 flags.go:64] FLAG: --healthz-port="10248"
Oct 30 21:52:59.566833 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.547135    4412 flags.go:64] FLAG: --help="false"
Oct 30 21:52:59.566833 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.547140    4412 flags.go:64] FLAG: --hostname-override="ip-10-0-131-202.us-west-1.compute.internal"
Oct 30 21:52:59.566833 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.547144    4412 flags.go:64] FLAG: --housekeeping-interval="10s"
Oct 30 21:52:59.566833 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.547147    4412 flags.go:64] FLAG: --http-check-frequency="20s"
Oct 30 21:52:59.566833 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.547149    4412 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Oct 30 21:52:59.566833 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.547153    4412 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Oct 30 21:52:59.566833 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.547156    4412 flags.go:64] FLAG: --image-gc-high-threshold="85"
Oct 30 21:52:59.566833 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.547159    4412 flags.go:64] FLAG: --image-gc-low-threshold="80"
Oct 30 21:52:59.566833 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.547162    4412 flags.go:64] FLAG: --image-service-endpoint=""
Oct 30 21:52:59.567413 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.547165    4412 flags.go:64] FLAG: --kernel-memcg-notification="false"
Oct 30 21:52:59.567413 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.547167    4412 flags.go:64] FLAG: --kube-api-burst="100"
Oct 30 21:52:59.567413 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.547170    4412 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Oct 30 21:52:59.567413 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.547173    4412 flags.go:64] FLAG: --kube-api-qps="50"
Oct 30 21:52:59.567413 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.547178    4412 flags.go:64] FLAG: --kube-reserved=""
Oct 30 21:52:59.567413 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.547181    4412 flags.go:64] FLAG: --kube-reserved-cgroup=""
Oct 30 21:52:59.567413 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.547184    4412 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Oct 30 21:52:59.567413 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.547187    4412 flags.go:64] FLAG: --kubelet-cgroups=""
Oct 30 21:52:59.567413 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.547190    4412 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Oct 30 21:52:59.567413 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.547193    4412 flags.go:64] FLAG: --lock-file=""
Oct 30 21:52:59.567413 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.547195    4412 flags.go:64] FLAG: --log-cadvisor-usage="false"
Oct 30 21:52:59.567413 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.547198    4412 flags.go:64] FLAG: --log-flush-frequency="5s"
Oct 30 21:52:59.567413 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.547201    4412 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Oct 30 21:52:59.567413 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.547207    4412 flags.go:64] FLAG: --log-json-split-stream="false"
Oct 30 21:52:59.567413 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.547210    4412 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Oct 30 21:52:59.567413 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.547212    4412 flags.go:64] FLAG: --log-text-split-stream="false"
Oct 30 21:52:59.567413 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.547215    4412 flags.go:64] FLAG: --logging-format="text"
Oct 30 21:52:59.567413 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.547219    4412 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Oct 30 21:52:59.567413 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.547223    4412 flags.go:64] FLAG: --make-iptables-util-chains="true"
Oct 30 21:52:59.567413 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.547225    4412 flags.go:64] FLAG: --manifest-url=""
Oct 30 21:52:59.567413 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.547228    4412 flags.go:64] FLAG: --manifest-url-header=""
Oct 30 21:52:59.567413 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.547233    4412 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Oct 30 21:52:59.567413 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.547236    4412 flags.go:64] FLAG: --max-open-files="1000000"
Oct 30 21:52:59.567413 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.547239    4412 flags.go:64] FLAG: --max-pods="110"
Oct 30 21:52:59.567413 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.547243    4412 flags.go:64] FLAG: --maximum-dead-containers="-1"
Oct 30 21:52:59.568074 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.547245    4412 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Oct 30 21:52:59.568074 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.547249    4412 flags.go:64] FLAG: --memory-manager-policy="None"
Oct 30 21:52:59.568074 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.547252    4412 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Oct 30 21:52:59.568074 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.547255    4412 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Oct 30 21:52:59.568074 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.547258    4412 flags.go:64] FLAG: --node-ip="0.0.0.0"
Oct 30 21:52:59.568074 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.547261    4412 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Oct 30 21:52:59.568074 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.547267    4412 flags.go:64] FLAG: --node-status-max-images="50"
Oct 30 21:52:59.568074 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.547270    4412 flags.go:64] FLAG: --node-status-update-frequency="10s"
Oct 30 21:52:59.568074 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.547273    4412 flags.go:64] FLAG: --oom-score-adj="-999"
Oct 30 21:52:59.568074 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.547276    4412 flags.go:64] FLAG: --pod-cidr=""
Oct 30 21:52:59.568074 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.547279    4412 flags.go:64] FLAG: --pod-manifest-path=""
Oct 30 21:52:59.568074 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.547284    4412 flags.go:64] FLAG: --pod-max-pids="-1"
Oct 30 21:52:59.568074 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.547287    4412 flags.go:64] FLAG: --pods-per-core="0"
Oct 30 21:52:59.568074 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.547290    4412 flags.go:64] FLAG: --port="10250"
Oct 30 21:52:59.568074 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.547293    4412 flags.go:64] FLAG: --protect-kernel-defaults="false"
Oct 30 21:52:59.568074 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.547296    4412 flags.go:64] FLAG: --provider-id="aws:///us-west-1a/i-02f3778ad02613422"
Oct 30 21:52:59.568074 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.547299    4412 flags.go:64] FLAG: --qos-reserved=""
Oct 30 21:52:59.568074 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.547301    4412 flags.go:64] FLAG: --read-only-port="10255"
Oct 30 21:52:59.568074 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.547304    4412 flags.go:64] FLAG: --register-node="true"
Oct 30 21:52:59.568074 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.547307    4412 flags.go:64] FLAG: --register-with-taints=""
Oct 30 21:52:59.568074 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.547311    4412 flags.go:64] FLAG: --registry-burst="10"
Oct 30 21:52:59.568074 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.547314    4412 flags.go:64] FLAG: --registry-qps="5"
Oct 30 21:52:59.568074 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.547317    4412 flags.go:64] FLAG: --reserved-cpus=""
Oct 30 21:52:59.568074 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.547319    4412 flags.go:64] FLAG: --reserved-memory=""
Oct 30 21:52:59.568074 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.547324    4412 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Oct 30 21:52:59.568074 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.547327    4412 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Oct 30 21:52:59.568737 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.547330    4412 flags.go:64] FLAG: --rotate-certificates="false"
Oct 30 21:52:59.568737 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.547333    4412 flags.go:64] FLAG: --rotate-server-certificates="false"
Oct 30 21:52:59.568737 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.547336    4412 flags.go:64] FLAG: --runonce="false"
Oct 30 21:52:59.568737 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.547339    4412 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Oct 30 21:52:59.568737 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.547341    4412 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Oct 30 21:52:59.568737 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.547344    4412 flags.go:64] FLAG: --seccomp-default="false"
Oct 30 21:52:59.568737 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.547347    4412 flags.go:64] FLAG: --serialize-image-pulls="true"
Oct 30 21:52:59.568737 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.547350    4412 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Oct 30 21:52:59.568737 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.547354    4412 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Oct 30 21:52:59.568737 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.547357    4412 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Oct 30 21:52:59.568737 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.547360    4412 flags.go:64] FLAG: --storage-driver-password="root"
Oct 30 21:52:59.568737 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.547363    4412 flags.go:64] FLAG: --storage-driver-secure="false"
Oct 30 21:52:59.568737 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.547366    4412 flags.go:64] FLAG: --storage-driver-table="stats"
Oct 30 21:52:59.568737 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.547369    4412 flags.go:64] FLAG: --storage-driver-user="root"
Oct 30 21:52:59.568737 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.547371    4412 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Oct 30 21:52:59.568737 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.547374    4412 flags.go:64] FLAG: --sync-frequency="1m0s"
Oct 30 21:52:59.568737 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.547377    4412 flags.go:64] FLAG: --system-cgroups=""
Oct 30 21:52:59.568737 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.547381    4412 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Oct 30 21:52:59.568737 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.547386    4412 flags.go:64] FLAG: --system-reserved-cgroup=""
Oct 30 21:52:59.568737 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.547389    4412 flags.go:64] FLAG: --tls-cert-file=""
Oct 30 21:52:59.568737 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.547392    4412 flags.go:64] FLAG: --tls-cipher-suites="[]"
Oct 30 21:52:59.568737 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.547396    4412 flags.go:64] FLAG: --tls-min-version=""
Oct 30 21:52:59.568737 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.547399    4412 flags.go:64] FLAG: --tls-private-key-file=""
Oct 30 21:52:59.568737 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.547402    4412 flags.go:64] FLAG: --topology-manager-policy="none"
Oct 30 21:52:59.568737 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.547405    4412 flags.go:64] FLAG: --topology-manager-policy-options=""
Oct 30 21:52:59.569473 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.547408    4412 flags.go:64] FLAG: --topology-manager-scope="container"
Oct 30 21:52:59.569473 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.547410    4412 flags.go:64] FLAG: --v="2"
Oct 30 21:52:59.569473 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.547415    4412 flags.go:64] FLAG: --version="false"
Oct 30 21:52:59.569473 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.547418    4412 flags.go:64] FLAG: --vmodule=""
Oct 30 21:52:59.569473 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.547422    4412 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Oct 30 21:52:59.569473 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.547427    4412 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Oct 30 21:52:59.569473 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.547550    4412 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 21:52:59.569473 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.547553    4412 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 21:52:59.569473 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.547557    4412 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 21:52:59.569473 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.547560    4412 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 21:52:59.569473 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.547563    4412 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 21:52:59.569473 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.547566    4412 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 21:52:59.569473 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.547568    4412 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 21:52:59.569473 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.547572    4412 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 21:52:59.569473 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.547576    4412 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 21:52:59.569473 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.547579    4412 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 21:52:59.569473 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.547582    4412 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 21:52:59.569473 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.547585    4412 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 21:52:59.569473 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.547588    4412 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 21:52:59.569473 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.547590    4412 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 21:52:59.569473 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.547593    4412 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 21:52:59.569977 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.547595    4412 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 21:52:59.569977 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.547598    4412 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 21:52:59.569977 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.547601    4412 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 21:52:59.569977 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.547605    4412 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 21:52:59.569977 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.547607    4412 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 21:52:59.569977 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.547610    4412 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 21:52:59.569977 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.547613    4412 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 21:52:59.569977 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.547617    4412 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 21:52:59.569977 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.547619    4412 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 21:52:59.569977 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.547622    4412 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 21:52:59.569977 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.547625    4412 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 21:52:59.569977 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.547627    4412 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 21:52:59.569977 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.547630    4412 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 21:52:59.569977 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.547633    4412 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 21:52:59.569977 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.547635    4412 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 21:52:59.569977 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.547637    4412 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 21:52:59.569977 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.547640    4412 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 21:52:59.569977 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.547643    4412 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 21:52:59.569977 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.547646    4412 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 21:52:59.569977 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.547648    4412 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 21:52:59.570508 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.547650    4412 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 21:52:59.570508 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.547653    4412 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 21:52:59.570508 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.547655    4412 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 21:52:59.570508 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.547658    4412 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 21:52:59.570508 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.547660    4412 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 21:52:59.570508 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.547663    4412 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 21:52:59.570508 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.547665    4412 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 21:52:59.570508 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.547667    4412 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 21:52:59.570508 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.547670    4412 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 21:52:59.570508 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.547673    4412 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 21:52:59.570508 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.547675    4412 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 21:52:59.570508 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.547678    4412 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 21:52:59.570508 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.547680    4412 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 21:52:59.570508 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.547683    4412 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 21:52:59.570508 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.547685    4412 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 21:52:59.570508 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.547688    4412 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 21:52:59.570508 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.547691    4412 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 21:52:59.570508 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.547694    4412 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 21:52:59.570508 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.547696    4412 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 21:52:59.570508 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.547699    4412 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 21:52:59.571178 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.547701    4412 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 21:52:59.571178 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.547703    4412 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 21:52:59.571178 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.547706    4412 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 21:52:59.571178 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.547708    4412 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 21:52:59.571178 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.547711    4412 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 21:52:59.571178 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.547714    4412 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 21:52:59.571178 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.547716    4412 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 21:52:59.571178 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.547719    4412 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 21:52:59.571178 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.547721    4412 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 21:52:59.571178 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.547724    4412 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 21:52:59.571178 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.547726    4412 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 21:52:59.571178 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.547729    4412 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 21:52:59.571178 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.547732    4412 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 21:52:59.571178 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.547734    4412 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 21:52:59.571178 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.547736    4412 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 21:52:59.571178 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.547739    4412 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 21:52:59.571178 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.547741    4412 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 21:52:59.571178 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.547744    4412 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 21:52:59.571178 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.547746    4412 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 21:52:59.571674 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.547749    4412 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 21:52:59.571674 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.547751    4412 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 21:52:59.571674 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.547754    4412 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 21:52:59.571674 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.547757    4412 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 21:52:59.571674 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.547759    4412 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 21:52:59.571674 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.547762    4412 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 21:52:59.571674 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.547764    4412 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 21:52:59.571674 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.547767    4412 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 21:52:59.571674 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.547771    4412 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 21:52:59.571674 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.547773    4412 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 21:52:59.571674 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.547776    4412 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 21:52:59.571674 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.547778    4412 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 21:52:59.571674 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.547781    4412 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 21:52:59.571674 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.547783    4412 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 21:52:59.571674 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.547786    4412 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 21:52:59.571674 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.547788    4412 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 21:52:59.571674 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.547791    4412 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 21:52:59.571674 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.547793    4412 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 21:52:59.571674 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.547796    4412 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 21:52:59.572210 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.547798    4412 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 21:52:59.572210 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.547801    4412 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 21:52:59.572210 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.547803    4412 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 21:52:59.572210 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.547806    4412 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 21:52:59.572210 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.547808    4412 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 21:52:59.572210 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.547811    4412 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 21:52:59.572210 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.547813    4412 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 21:52:59.572210 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.547816    4412 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 21:52:59.572210 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.547818    4412 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 21:52:59.572210 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.547821    4412 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 21:52:59.572210 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.547823    4412 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 21:52:59.572210 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.547826    4412 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 21:52:59.572210 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.547828    4412 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 21:52:59.572210 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.547853    4412 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 30 21:52:59.572210 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.547887    4412 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 21:52:59.572210 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.547891    4412 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 21:52:59.572948 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.547894    4412 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 21:52:59.572948 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.547897    4412 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 21:52:59.572948 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.547899    4412 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 21:52:59.572948 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.547902    4412 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 21:52:59.572948 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.547905    4412 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 21:52:59.572948 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.547909    4412 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 21:52:59.572948 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.547911    4412 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 21:52:59.572948 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.547923    4412 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 21:52:59.572948 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.547928    4412 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 21:52:59.572948 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.547932    4412 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 21:52:59.572948 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.547936    4412 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 21:52:59.572948 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.547939    4412 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 21:52:59.572948 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.547941    4412 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 21:52:59.572948 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.547944    4412 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 21:52:59.572948 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.547946    4412 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 21:52:59.572948 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.547949    4412 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 21:52:59.572948 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.547951    4412 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 21:52:59.572948 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.547954    4412 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 21:52:59.572948 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.547956    4412 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 21:52:59.572948 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.547959    4412 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 21:52:59.573786 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.547961    4412 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 21:52:59.573786 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.547964    4412 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 21:52:59.573786 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.547966    4412 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 21:52:59.573786 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.547969    4412 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 21:52:59.573786 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.547971    4412 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 21:52:59.573786 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.547974    4412 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 21:52:59.573786 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.547976    4412 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 21:52:59.573786 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.547978    4412 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 21:52:59.573786 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.547981    4412 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 21:52:59.573786 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.547983    4412 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 21:52:59.573786 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.547985    4412 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 21:52:59.573786 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.547988    4412 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 21:52:59.573786 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.547991    4412 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 21:52:59.573786 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.548001    4412 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 21:52:59.573786 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.548004    4412 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 21:52:59.573786 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.548008    4412 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 21:52:59.573786 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.548011    4412 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 21:52:59.573786 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.548015    4412 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 21:52:59.573786 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.548017    4412 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 21:52:59.573786 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.548020    4412 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 21:52:59.574416 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.548023    4412 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 21:52:59.574416 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.548026    4412 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 21:52:59.574416 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.548028    4412 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 21:52:59.574416 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.548031    4412 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 21:52:59.574416 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.548033    4412 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 21:52:59.574416 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.548036    4412 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 21:52:59.574416 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.548039    4412 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 21:52:59.574416 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.548042    4412 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 21:52:59.574416 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.548044    4412 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 21:52:59.574416 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.548047    4412 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 21:52:59.574416 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.548049    4412 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 21:52:59.574416 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.548052    4412 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 21:52:59.574416 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.548054    4412 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 21:52:59.574416 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.548057    4412 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 21:52:59.574416 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.548059    4412 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 21:52:59.574416 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.548062    4412 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 21:52:59.574416 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.548064    4412 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 21:52:59.574416 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.548066    4412 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 21:52:59.574416 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.548069    4412 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 21:52:59.574416 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.548071    4412 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 21:52:59.574976 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.548073    4412 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 21:52:59.574976 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.548076    4412 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 21:52:59.574976 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.548078    4412 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 21:52:59.574976 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.548081    4412 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 21:52:59.574976 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.548083    4412 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 21:52:59.574976 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.548086    4412 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 21:52:59.574976 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.548089    4412 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 21:52:59.574976 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.548093    4412 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 21:52:59.574976 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.548095    4412 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 21:52:59.574976 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.548099    4412 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 21:52:59.574976 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.548102    4412 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 21:52:59.574976 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.548104    4412 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 21:52:59.574976 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.548107    4412 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 21:52:59.574976 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.548109    4412 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 21:52:59.574976 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.548112    4412 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 21:52:59.574976 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.548114    4412 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 21:52:59.574976 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.548133    4412 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 21:52:59.574976 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.548136    4412 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 21:52:59.574976 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.548138    4412 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 21:52:59.575501 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.548141    4412 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 21:52:59.575501 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.548143    4412 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 21:52:59.575501 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.548146    4412 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 21:52:59.575501 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.548148    4412 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 21:52:59.575501 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.548151    4412 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 21:52:59.575501 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.548153    4412 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 21:52:59.575501 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.548156    4412 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 21:52:59.575501 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.548158    4412 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 21:52:59.575501 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.548161    4412 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 21:52:59.575501 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.548163    4412 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 21:52:59.575501 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.548166    4412 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 21:52:59.575501 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.548168    4412 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 21:52:59.575501 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.548170    4412 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 21:52:59.575501 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.548173    4412 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 21:52:59.575501 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.548175    4412 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 21:52:59.575501 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.548177    4412 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 21:52:59.575501 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.548180    4412 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 21:52:59.575501 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.548182    4412 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 21:52:59.575501 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.548185    4412 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 21:52:59.576027 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.548188    4412 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 21:52:59.576027 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.548192    4412 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 21:52:59.576027 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.548197    4412 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 21:52:59.576027 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.548201    4412 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 21:52:59.576027 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.548204    4412 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 21:52:59.576027 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.548207    4412 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 21:52:59.576027 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.552462    4412 server.go:525] "Kubelet version" kubeletVersion="v1.35.9"
Oct 30 21:52:59.576027 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.552476    4412 server.go:527] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Oct 30 21:52:59.576027 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.552494    4412 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Oct 30 21:52:59.576027 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.552499    4412 watchdog_linux.go:138] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Oct 30 21:52:59.576027 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.552571    4412 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 21:52:59.576027 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.552576    4412 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 21:52:59.576027 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.552579    4412 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 21:52:59.576027 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.552582    4412 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 21:52:59.576027 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.552585    4412 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 21:52:59.576027 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.552589    4412 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 21:52:59.576027 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.552592    4412 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 21:52:59.576027 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.552594    4412 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 21:52:59.576027 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.552597    4412 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 21:52:59.576554 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.552600    4412 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 21:52:59.576554 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.552602    4412 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 21:52:59.576554 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.552605    4412 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 21:52:59.576554 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.552607    4412 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 21:52:59.576554 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.552610    4412 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 21:52:59.576554 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.552613    4412 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 21:52:59.576554 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.552615    4412 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 21:52:59.576554 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.552618    4412 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 21:52:59.576554 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.552621    4412 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 21:52:59.576554 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.552623    4412 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 21:52:59.576554 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.552626    4412 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 21:52:59.576554 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.552628    4412 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 21:52:59.576554 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.552632    4412 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 21:52:59.576554 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.552637    4412 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 21:52:59.576554 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.552641    4412 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 21:52:59.576554 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.552644    4412 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 21:52:59.576554 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.552647    4412 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 21:52:59.576554 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.552649    4412 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 21:52:59.576554 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.552652    4412 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 21:52:59.577067 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.552655    4412 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 21:52:59.577067 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.552657    4412 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 21:52:59.577067 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.552660    4412 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 21:52:59.577067 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.552662    4412 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 21:52:59.577067 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.552665    4412 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 21:52:59.577067 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.552668    4412 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 21:52:59.577067 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.552670    4412 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 21:52:59.577067 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.552673    4412 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 21:52:59.577067 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.552675    4412 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 21:52:59.577067 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.552677    4412 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 21:52:59.577067 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.552680    4412 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 21:52:59.577067 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.552683    4412 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 21:52:59.577067 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.552685    4412 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 21:52:59.577067 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.552688    4412 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 21:52:59.577067 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.552690    4412 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 21:52:59.577067 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.552692    4412 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 21:52:59.577067 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.552695    4412 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 21:52:59.577067 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.552697    4412 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 21:52:59.577067 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.552700    4412 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 21:52:59.577067 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.552702    4412 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 21:52:59.577608 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.552705    4412 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 21:52:59.577608 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.552707    4412 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 21:52:59.577608 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.552710    4412 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 21:52:59.577608 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.552712    4412 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 21:52:59.577608 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.552715    4412 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 21:52:59.577608 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.552717    4412 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 21:52:59.577608 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.552720    4412 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 21:52:59.577608 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.552723    4412 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 21:52:59.577608 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.552731    4412 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 21:52:59.577608 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.552734    4412 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 21:52:59.577608 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.552737    4412 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 21:52:59.577608 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.552739    4412 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 21:52:59.577608 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.552742    4412 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 21:52:59.577608 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.552744    4412 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 21:52:59.577608 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.552747    4412 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 21:52:59.577608 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.552749    4412 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 21:52:59.577608 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.552752    4412 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 21:52:59.577608 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.552754    4412 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 21:52:59.577608 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.552756    4412 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 21:52:59.577608 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.552759    4412 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 21:52:59.578112 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.552761    4412 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 21:52:59.578112 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.552764    4412 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 21:52:59.578112 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.552766    4412 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 21:52:59.578112 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.552768    4412 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 21:52:59.578112 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.552771    4412 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 21:52:59.578112 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.552773    4412 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 21:52:59.578112 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.552775    4412 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 21:52:59.578112 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.552778    4412 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 21:52:59.578112 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.552780    4412 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 21:52:59.578112 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.552783    4412 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 21:52:59.578112 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.552785    4412 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 21:52:59.578112 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.552787    4412 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 21:52:59.578112 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.552790    4412 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 21:52:59.578112 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.552792    4412 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 21:52:59.578112 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.552795    4412 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 21:52:59.578112 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.552798    4412 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 21:52:59.578112 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.552800    4412 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 21:52:59.578112 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.552803    4412 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 21:52:59.578112 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.552805    4412 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 21:52:59.578112 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.552807    4412 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 21:52:59.578736 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.552810    4412 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 21:52:59.578736 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.552818    4412 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 21:52:59.578736 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.552821    4412 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 21:52:59.578736 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.552823    4412 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 21:52:59.578736 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.552827    4412 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 21:52:59.578736 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.552831    4412 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 21:52:59.578736 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.552834    4412 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 21:52:59.578736 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.552837    4412 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 21:52:59.578736 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.552839    4412 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 21:52:59.578736 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.552841    4412 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 21:52:59.578736 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.552844    4412 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 21:52:59.578736 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.552846    4412 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 21:52:59.578736 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.552849    4412 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 21:52:59.578736 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.552851    4412 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 21:52:59.578736 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.552854    4412 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 21:52:59.578736 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.552856    4412 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 21:52:59.578736 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.552859    4412 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 21:52:59.578736 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.552861    4412 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 21:52:59.579246 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.552888    4412 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 30 21:52:59.579246 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.553083    4412 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 21:52:59.579246 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.553089    4412 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 21:52:59.579246 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.553093    4412 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 21:52:59.579246 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.553096    4412 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 21:52:59.579246 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.553099    4412 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 21:52:59.579246 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.553104    4412 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 21:52:59.579246 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.553107    4412 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 21:52:59.579246 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.553110    4412 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 21:52:59.579246 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.553113    4412 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 21:52:59.579246 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.553132    4412 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 21:52:59.579246 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.553136    4412 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 21:52:59.579246 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.553140    4412 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 21:52:59.579246 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.553142    4412 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 21:52:59.579246 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.553145    4412 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 21:52:59.579627 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.553147    4412 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 21:52:59.579627 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.553158    4412 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 21:52:59.579627 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.553161    4412 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 21:52:59.579627 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.553163    4412 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 21:52:59.579627 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.553166    4412 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 21:52:59.579627 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.553168    4412 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 21:52:59.579627 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.553171    4412 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 21:52:59.579627 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.553175    4412 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 21:52:59.579627 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.553178    4412 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 21:52:59.579627 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.553181    4412 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 21:52:59.579627 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.553184    4412 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 21:52:59.579627 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.553186    4412 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 21:52:59.579627 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.553189    4412 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 21:52:59.579627 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.553191    4412 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 21:52:59.579627 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.553194    4412 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 21:52:59.579627 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.553196    4412 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 21:52:59.579627 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.553198    4412 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 21:52:59.579627 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.553201    4412 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 21:52:59.579627 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.553203    4412 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 21:52:59.580094 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.553206    4412 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 21:52:59.580094 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.553209    4412 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 21:52:59.580094 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.553211    4412 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 21:52:59.580094 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.553214    4412 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 21:52:59.580094 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.553218    4412 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 21:52:59.580094 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.553220    4412 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 21:52:59.580094 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.553223    4412 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 21:52:59.580094 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.553225    4412 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 21:52:59.580094 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.553228    4412 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 21:52:59.580094 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.553230    4412 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 21:52:59.580094 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.553233    4412 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 21:52:59.580094 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.553235    4412 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 21:52:59.580094 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.553238    4412 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 21:52:59.580094 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.553240    4412 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 21:52:59.580094 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.553243    4412 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 21:52:59.580094 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.553251    4412 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 21:52:59.580094 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.553254    4412 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 21:52:59.580094 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.553256    4412 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 21:52:59.580094 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.553259    4412 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 21:52:59.580094 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.553261    4412 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 21:52:59.580693 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.553264    4412 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 21:52:59.580693 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.553266    4412 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 21:52:59.580693 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.553269    4412 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 21:52:59.580693 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.553271    4412 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 21:52:59.580693 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.553274    4412 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 21:52:59.580693 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.553276    4412 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 21:52:59.580693 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.553279    4412 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 21:52:59.580693 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.553281    4412 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 21:52:59.580693 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.553284    4412 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 21:52:59.580693 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.553287    4412 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 21:52:59.580693 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.553291    4412 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 21:52:59.580693 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.553293    4412 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 21:52:59.580693 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.553296    4412 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 21:52:59.580693 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.553298    4412 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 21:52:59.580693 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.553301    4412 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 21:52:59.580693 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.553303    4412 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 21:52:59.580693 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.553307    4412 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 21:52:59.580693 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.553309    4412 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 21:52:59.580693 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.553312    4412 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 21:52:59.580693 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.553314    4412 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 21:52:59.581208 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.553317    4412 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 21:52:59.581208 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.553319    4412 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 21:52:59.581208 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.553322    4412 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 21:52:59.581208 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.553324    4412 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 21:52:59.581208 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.553327    4412 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 21:52:59.581208 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.553329    4412 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 21:52:59.581208 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.553332    4412 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 21:52:59.581208 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.553334    4412 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 21:52:59.581208 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.553337    4412 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 21:52:59.581208 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.553340    4412 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 21:52:59.581208 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.553343    4412 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 21:52:59.581208 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.553345    4412 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 21:52:59.581208 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.553348    4412 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 21:52:59.581208 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.553350    4412 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 21:52:59.581208 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.553353    4412 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 21:52:59.581208 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.553355    4412 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 21:52:59.581208 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.553358    4412 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 21:52:59.581208 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.553360    4412 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 21:52:59.581208 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.553363    4412 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 21:52:59.581659 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.553365    4412 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 21:52:59.581659 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.553368    4412 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 21:52:59.581659 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.553370    4412 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 21:52:59.581659 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.553374    4412 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 21:52:59.581659 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.553376    4412 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 21:52:59.581659 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.553379    4412 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 21:52:59.581659 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.553382    4412 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 21:52:59.581659 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.553384    4412 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 21:52:59.581659 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.553386    4412 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 21:52:59.581659 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.553390    4412 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 21:52:59.581659 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.553393    4412 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 21:52:59.581659 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.553395    4412 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 21:52:59.581659 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.553398    4412 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 21:52:59.581659 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.553400    4412 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 21:52:59.581659 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.553425    4412 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 30 21:52:59.581659 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.553459    4412 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 21:52:59.582054 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.553463    4412 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 21:52:59.582054 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.553467    4412 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 21:52:59.582054 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.553470    4412 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 21:52:59.582054 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.553473    4412 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 21:52:59.582054 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.553475    4412 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 21:52:59.582054 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.553478    4412 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 21:52:59.582054 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.553481    4412 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 21:52:59.582054 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.553485    4412 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 21:52:59.582054 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.553488    4412 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 21:52:59.582054 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.553490    4412 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 21:52:59.582054 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.553493    4412 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 21:52:59.582054 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.553495    4412 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 21:52:59.582054 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.553498    4412 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 21:52:59.582054 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.553500    4412 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 21:52:59.582054 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.553503    4412 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 21:52:59.582054 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.553505    4412 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 21:52:59.582054 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.553508    4412 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 21:52:59.582054 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.553510    4412 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 21:52:59.582054 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.553513    4412 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 21:52:59.582054 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.553516    4412 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 21:52:59.582793 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.553519    4412 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 21:52:59.582793 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.553521    4412 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 21:52:59.582793 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.553523    4412 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 21:52:59.582793 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.553526    4412 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 21:52:59.582793 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.553528    4412 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 21:52:59.582793 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.553531    4412 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 21:52:59.582793 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.553535    4412 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 21:52:59.582793 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.553538    4412 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 21:52:59.582793 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.553540    4412 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 21:52:59.582793 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.553543    4412 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 21:52:59.582793 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.553545    4412 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 21:52:59.582793 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.553548    4412 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 21:52:59.582793 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.553551    4412 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 21:52:59.582793 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.553553    4412 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 21:52:59.582793 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.553556    4412 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 21:52:59.582793 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.553559    4412 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 21:52:59.582793 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.553561    4412 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 21:52:59.582793 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.553564    4412 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 21:52:59.582793 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.553566    4412 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 21:52:59.583582 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.553569    4412 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 21:52:59.583582 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.553572    4412 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 21:52:59.583582 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.553575    4412 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 21:52:59.583582 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.553577    4412 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 21:52:59.583582 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.553580    4412 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 21:52:59.583582 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.553582    4412 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 21:52:59.583582 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.553584    4412 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 21:52:59.583582 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.553587    4412 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 21:52:59.583582 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.553590    4412 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 21:52:59.583582 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.553593    4412 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 21:52:59.583582 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.553597    4412 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 21:52:59.583582 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.553600    4412 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 21:52:59.583582 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.553605    4412 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 21:52:59.583582 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.553608    4412 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 21:52:59.583582 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.553610    4412 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 21:52:59.583582 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.553613    4412 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 21:52:59.583582 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.553617    4412 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 21:52:59.583582 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.553620    4412 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 21:52:59.583582 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.553622    4412 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 21:52:59.582868 ip-10-0-131-202 systemd[1]: Started Kubernetes Kubelet.
Oct 30 21:52:59.584594 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.553626    4412 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 21:52:59.584594 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.553629    4412 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 21:52:59.584594 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.553631    4412 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 21:52:59.584594 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.553634    4412 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 21:52:59.584594 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.553636    4412 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 21:52:59.584594 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.553639    4412 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 21:52:59.584594 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.553641    4412 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 21:52:59.584594 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.553644    4412 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 21:52:59.584594 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.553646    4412 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 21:52:59.584594 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.553649    4412 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 21:52:59.584594 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.553651    4412 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 21:52:59.584594 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.553655    4412 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 21:52:59.584594 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.553659    4412 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 21:52:59.584594 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.553662    4412 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 21:52:59.584594 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.553665    4412 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 21:52:59.584594 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.553667    4412 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 21:52:59.584594 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.553670    4412 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 21:52:59.584594 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.553672    4412 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 21:52:59.584594 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.553675    4412 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 21:52:59.585420 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.553677    4412 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 21:52:59.585420 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.553680    4412 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 21:52:59.585420 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.553682    4412 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 21:52:59.585420 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.553685    4412 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 21:52:59.585420 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.553687    4412 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 21:52:59.585420 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.553690    4412 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 21:52:59.585420 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.553695    4412 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 21:52:59.585420 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.553698    4412 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 21:52:59.585420 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.553700    4412 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 21:52:59.585420 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.553703    4412 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 21:52:59.585420 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.553705    4412 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 21:52:59.585420 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.553708    4412 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 21:52:59.585420 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.553710    4412 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 21:52:59.585420 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.553714    4412 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 21:52:59.585420 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.553718    4412 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 21:52:59.585420 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.553721    4412 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 21:52:59.585420 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.553723    4412 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 21:52:59.585420 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.553726    4412 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 21:52:59.585420 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.553728    4412 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 21:52:59.585420 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.553731    4412 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 21:52:59.586489 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.553733    4412 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 21:52:59.586489 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.553736    4412 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 21:52:59.586489 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.553739    4412 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 21:52:59.586489 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.553741    4412 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 21:52:59.586489 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.553744    4412 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 21:52:59.586489 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.553746    4412 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 21:52:59.586489 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.553749    4412 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 21:52:59.586489 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:52:59.553751    4412 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 21:52:59.586489 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.553799    4412 server.go:957] "Client rotation is on, will bootstrap in background"
Oct 30 21:52:59.586489 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.554774    4412 bootstrap.go:85] "Current kubeconfig file contents are still valid, no bootstrap necessary"
Oct 30 21:52:59.586489 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.554830    4412 certificate_store.go:147] "Loading cert/key pair from a file" filePath="/var/lib/kubelet/pki/kubelet-client-current.pem"
Oct 30 21:52:59.586489 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.555013    4412 server.go:1014] "Starting client certificate rotation"
Oct 30 21:52:59.586489 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.555143    4412 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Oct 30 21:52:59.586489 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.555186    4412 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-10-29 21:47:20 +0000 UTC" deadline="2028-06-25 03:34:09.377344672 +0000 UTC"
Oct 30 21:52:59.586489 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.555213    4412 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14477h41m9.822133608s"
Oct 30 21:52:59.586489 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.555461    4412 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Oct 30 21:52:59.587209 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.556532    4412 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Oct 30 21:52:59.587209 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.559107    4412 log.go:25] "Validated CRI v1 runtime API"
Oct 30 21:52:59.587209 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.560569    4412 log.go:25] "Validated CRI v1 image API"
Oct 30 21:52:59.587209 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.561232    4412 server.go:1424] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Oct 30 21:52:59.587209 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.562613    4412 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 807d4c8b-be93-4a35-b656-6e19d4292167:/dev/nvme0n1p3 96540798-6f1c-4a39-a1ec-8fe598f5180f:/dev/nvme0n1p4]
Oct 30 21:52:59.587436 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.562628    4412 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:24 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:26 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/3970e1be64d4877f260bff03ca6a43749580fdc4c5a49479929398c86576395e/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/3970e1be64d4877f260bff03ca6a43749580fdc4c5a49479929398c86576395e/userdata/shm major:0 minor:98 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/3f2b3c7943e258b620f38697047406863aa88efda8d5faada68af4c4e955aeb5/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/3f2b3c7943e258b620f38697047406863aa88efda8d5faada68af4c4e955aeb5/userdata/shm major:0 minor:82 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/3f3feb24d73c361bf3c8b6abc3c8ab7edc3aa0f9549ddbec02315f3dd7ed0b0c/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/3f3feb24d73c361bf3c8b6abc3c8ab7edc3aa0f9549ddbec02315f3dd7ed0b0c/userdata/shm major:0 minor:210 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/437d7c0fb64e2862943d48eb6c7731f8d740ca3387b8624fad4e28e5b29e1511/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/437d7c0fb64e2862943d48eb6c7731f8d740ca3387b8624fad4e28e5b29e1511/userdata/shm major:0 minor:63 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/5a0d9b46d476286e5b9f13aa9ed77125183273c6d32c4e1fb87b7608b8e29b35/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/5a0d9b46d476286e5b9f13aa9ed77125183273c6d32c4e1fb87b7608b8e29b35/userdata/shm major:0 minor:230 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/693c46ffe0fe055558dabec72ff633a79b91a6e9ec57f3e27c1c31178984aaa7/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/693c46ffe0fe055558dabec72ff633a79b91a6e9ec57f3e27c1c31178984aaa7/userdata/shm major:0 minor:60 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/6e596f1f222be642b03bca48eb0148c607deb4bab720ed68a09e44e31f2c0dea/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/6e596f1f222be642b03bca48eb0148c607deb4bab720ed68a09e44e31f2c0dea/userdata/shm major:0 minor:86 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/705914d7dce2552b00ce3ab62bd974e25578a0cc87f776136f313b6d4f04b10e/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/705914d7dce2552b00ce3ab62bd974e25578a0cc87f776136f313b6d4f04b10e/userdata/shm major:0 minor:90 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/9751c27f53d33b0e6f09e8d175690fc83db3fe0e1f08314102e30aca663fe0c9/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/9751c27f53d33b0e6f09e8d175690fc83db3fe0e1f08314102e30aca663fe0c9/userdata/shm major:0 minor:62 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/9e1ebfcd253a8ad808556ec9db0d8260ed7f2447b3ad5380483f2d6e13239593/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/9e1ebfcd253a8ad808556ec9db0d8260ed7f2447b3ad5380483f2d6e13239593/userdata/shm major:0 minor:94 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/a9887498d2723b7a9e696b07db4f758f0c7f19897e1b590ceb5739927bce13cb/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/a9887498d2723b7a9e696b07db4f758f0c7f19897e1b590ceb5739927bce13cb/userdata/shm major:0 minor:61 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/c8ab0219dae8058a1a0ad4e07bca8ce1cc695a209049fe6f3e048c5e56ae3da0/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/c8ab0219dae8058a1a0ad4e07bca8ce1cc695a209049fe6f3e048c5e56ae3da0/userdata/shm major:0 minor:202 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/da317ccc84703e328e24540d39e76dcf76ce091d7baad369756dba3796160cfb/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/da317ccc84703e328e24540d39e76dcf76ce091d7baad369756dba3796160cfb/userdata/shm major:0 minor:64 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/eecea248f97ac1f5faa4ef08a1b96061cee2cce9cac0767037b2d544ff1a8c1d/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/eecea248f97ac1f5faa4ef08a1b96061cee2cce9cac0767037b2d544ff1a8c1d/userdata/shm major:0 minor:208 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:32 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/164bafd8-ff17-46a2-afc0-dd01fa1aaeb6/volumes/kubernetes.io~projected/kube-api-access-78t6j:{mountpoint:/var/lib/kubelet/pods/164bafd8-ff17-46a2-afc0-dd01fa1aaeb6/volumes/kubernetes.io~projected/kube-api-access-78t6j major:0 minor:57 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/2345ff5b-db97-4855-8adb-59b9a780615e/volumes/kubernetes.io~projected/kube-api-access-c622t:{mountpoint:/var/lib/kubelet/pods/2345ff5b-db97-4855-8adb-59b9a780615e/volumes/kubernetes.io~projected/kube-api-access-c622t major:0 minor:197 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/2345ff5b-db97-4855-8adb-59b9a780615e/volumes/kubernetes.io~secret/cert:{mountpoint:/var/lib/kubelet/pods/2345ff5b-db97-4855-8adb-59b9a780615e/volumes/kubernetes.io~secret/cert major:0 minor:45 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/25590e1f-99e3-4469-8a69-39cd30317c28/volumes/kubernetes.io~secret/agent-certs:{mountpoint:/var/lib/kubelet/pods/25590e1f-99e3-4469-8a69-39cd30317c28/volumes/kubernetes.io~secret/agent-certs major:0 minor:50 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/386f9569-a5f5-4a9e-beee-3005994186cd/volumes/kubernetes.io~projected/kube-api-access-tg4f9:{mountpoint:/var/lib/kubelet/pods/386f9569-a5f5-4a9e-beee-3005994186cd/volumes/kubernetes.io~projected/kube-api-access-tg4f9 major:0 minor:59 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/4a03a836-506f-404e-9c55-c26d779cd4ee/volumes/kubernetes.io~projected/kube-api-access-dtsnb:{mountpoint:/var/lib/kubelet/pods/4a03a836-506f-404e-9c55-c26d779cd4ee/volumes/kubernetes.io~projected/kube-api-access-dtsnb major:0 minor:201 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/7be00c31-281f-4d1c-8e12-1bf74bbe7cbd/volumes/kubernetes.io~projected/kube-api-access-6j99v:{mountpoint:/var/lib/kubelet/pods/7be00c31-281f-4d1c-8e12-1bf74bbe7cbd/volumes/kubernetes.io~projected/kube-api-access-6j99v major:0 minor:207 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/7e6b0df0-07eb-4301-adaa-1a6514639d9c/volumes/kubernetes.io~projected/kube-api-access-fsf4l:{mountpoint:/var/lib/kubelet/pods/7e6b0df0-07eb-4301-adaa-1a6514639d9c/volumes/kubernetes.io~projected/kube-api-access-fsf4l major:0 minor:229 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/8806f1e1-0057-4738-8fbc-7e503e1f91ca/volumes/kubernetes.io~projected/kube-api-access-fwsxm:{mountpoint:/var/lib/kubelet/pods/8806f1e1-0057-4738-8fbc-7e503e1f91ca/volumes/kubernetes.io~projected/kube-api-access-fwsxm major:0 minor:58 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/982efb96-3613-4ef2-ac51-ed3f40745a32/volumes/kubernetes.io~empty-dir/etc-tuned:{mountpoint:/var/lib/kubelet/pods/982efb96-3613-4ef2-ac51-ed3f40745a32/volumes/kubernetes.io~empty-dir/etc-tuned major:0 minor:47 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/982efb96-3613-4ef2-ac51-ed3f40745a32/volumes/kubernetes.io~empty-dir/tmp:{mountpoint:/var/lib/kubelet/pods/982efb96-3613-4ef2-ac51-ed3f40745a32/volumes/kubernetes.io~empty-dir/tmp major:0 minor:49 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/982efb96-3613-4ef2-ac51-ed3f40745a32/volumes/kubernetes.io~projected/kube-api-access-2pwzh:{mountpoint:/var/lib/kubelet/pods/982efb96-3613-4ef2-ac51-ed3f40745a32/volumes/kubernetes.io~projected/kube-api-access-2pwzh major:0 minor:53 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/a4ff4cce-671b-4136-8b63-b22148c0db4e/volumes/kubernetes.io~projected/kube-api-access-6r8d9:{mountpoint:/var/lib/kubelet/pods/a4ff4cce-671b-4136-8b63-b22148c0db4e/volumes/kubernetes.io~projected/kube-api-access-6r8d9 major:0 minor:54 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/b1a02001-9121-47bd-ae96-b39196ad2357/volumes/kubernetes.io~projected/kube-api-access-vsg28:{mountpoint:/var/lib/kubelet/pods/b1a02001-9121-47bd-ae96-b39196ad2357/volumes/kubernetes.io~projected/kube-api-access-vsg28 major:0 minor:55 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/bf0e36d0-87bd-4e1e-be40-8a3a0f856987/volume-subpaths/run-systemd/ovnkube-controller/6:{mountpoint:/var/lib/kubelet/pods/bf0e36d0-87bd-4e1e-be40-8a3a0f856987/volume-subpaths/run-systemd/ovnkube-controller/6 major:0 minor:26 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/bf0e36d0-87bd-4e1e-be40-8a3a0f856987/volumes/kubernetes.io~projected/kube-api-access-ctg2b:{mountpoint:/var/lib/kubelet/pods/bf0e36d0-87bd-4e1e-be40-8a3a0f856987/volumes/kubernetes.io~projected/kube-api-access-ctg2b major:0 minor:52 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/bf0e36d0-87bd-4e1e-be40-8a3a0f856987/volumes/kubernetes.io~secret/ovn-node-metrics-cert:{mountpoint:/var/lib/kubelet/pods/bf0e36d0-87bd-4e1e-be40-8a3a0f856987/volumes/kubernetes.io~secret/ovn-node-metrics-cert major:0 minor:48 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/c590d5fe-57de-4905-90fe-8527b215ce94/volumes/kubernetes.io~projected/kube-api-access-fq868:{mountpoint:/var/lib/kubelet/pods/c590d5fe-57de-4905-90fe-8527b215ce94/volumes/kubernetes.io~projected/kube-api-access-fq868 major:0 minor:51 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/d17b7754-d1d4-4952-97c7-7a04c714b48c/volumes/kubernetes.io~secret/original-pull-secret:{mountpoint:/var/lib/kubelet/pods/d17b7754-d1d4-4952-97c7-7a04c714b48c/volumes/kubernetes.io~secret/original-pull-secret major:0 minor:206 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/db073c3b-7cee-4e22-8c11-209bbeb21a96/volumes/kubernetes.io~projected/kube-api-access-vd7d5:{mountpoint:/var/lib/kubelet/pods/db073c3b-7cee-4e22-8c11-209bbeb21a96/volumes/kubernetes.io~projected/kube-api-access-vd7d5 major:0 minor:224 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/dffd7a70-b7f3-4e1b-b3ab-74c5a574cab2/volumes/kubernetes.io~projected/kube-api-access-hrlj5:{mountpoint:/var/lib/kubelet/pods/dffd7a70-b7f3-4e1b-b3ab-74c5a574cab2/volumes/kubernetes.io~projected/kube-api-access-hrlj5 major:0 minor:56 fsType:tmpfs blockSize:0} composefs_0-34:{mountpoint:/ major:0 minor:34 fsType:overlay blockSize:0} overlay_0-100:{mountpoint:/var/lib/containers/storage/overlay/a3b3e585572bdad4fc52a30d302c153d697f44bb9b20a93858fe940f6c129b05/merged major:0 minor:100 fsType:overlay blockSize:0} overlay_0-102:{mountpoint:/var/lib/containers/storage/overlay/30c14b50bb78739420a5939ccd8d3f194a728136078bcdef9d275259f9bce4c2/merged major:0 minor:102 fsType:overlay blockSize:0} overlay_0-104:{mountpoint:/var/lib/containers/storage/overlay/8c84ef117cf9ce3e4884d53c37dbc169679e7ab0993cccc8e28b5052e6bd7688/merged major:0 minor:104 fsType:overlay blockSize:0} overlay_0-109:{mountpoint:/var/lib/containers/storage/overlay/1db2973c01cc9d18de43615d55e1a616209434f32ca87c354a1ad84f48c8891d/merged major:0 minor:109 fsType:overlay blockSize:0} overlay_0-111:{mountpoint:/var/lib/containers/storage/overlay/1b7df3e1fca16e9e5143970db32140db176305cef0393d3d27033c397f2b3856/merged major:0 minor:111 fsType:overlay blockSize:0} overlay_0-119:{mountpoint:/var/lib/containers/storage/overlay/cde00307ef84661e2a61c8d42174d9d62af67889638dedb1ae94b624be730e26/merged major:0 minor:119 fsType:overlay blockSize:0} overlay_0-124:{mountpoint:/var/lib/containers/storage/overlay/cf452a58e936b8e5190ba3b9a602228099f297307f1e6724bacbdfdd0b85c776/merged major:0 minor:124 fsType:overlay blockSize:0} overlay_0-127:{mountpoint:/var/lib/containers/storage/overlay/a4ec8c3ecf6a490bf2a48a4311672a2fa2c34fab8eb27be9cc9fb4fc3477ce37/merged major:0 minor:127 fsType:overlay blockSize:0} overlay_0-132:{mountpoint:/var/lib/containers/storage/overlay/179004e89047568e2384bed27d15f82c468a09057a461aebced7d4762ef2627a/merged major:0 minor:132 fsType:overlay blockSize:0} overlay_0-137:{mountpoint:/var/lib/containers/storage/overlay/2c0d6e2f7ade86aa000a387876d880c5465e0259391da8c4df5fe02a755aace7/merged major:0 minor:137 fsType:overlay blockSize:0} overlay_0-139:{mountpoint:/var/lib/containers/storage/overlay/31d6f6b2d2de14f5a4b1483beb91d1ee3b230f0ab0b4026db8681017e53a806e/merged major:0 minor:139 fsType:overlay blockSize:0} overlay_0-147:{mountpoint:/var/lib/containers/storage/overlay/9e3763b0d818d14720839df0cb6ee2a803c8be5db624d22e974c2c60edba3924/merged major:0 minor:147 fsType:overlay blockSize:0} overlay_0-149:{mountpoint:/var/lib/containers/storage/overlay/81582472553edd7b5f9a9fbb80d534446da8d601bf77a7292428b2401ec4e003/merged major:0 minor:149 fsType:overlay blockSize:0} overlay_0-157:{mountpoint:/var/lib/containers/storage/overlay/98af1ac1954325d44cc40dde12c93a68e7ea487e9f62b017ff4973460cd2bd93/merged major:0 minor:157 fsType:overlay blockSize:0} overlay_0-162:{mountpoint:/var/lib/containers/storage/overlay/f1bc9edb76d4a22135c6d77e45a5316d554eb4b57fa1cc74398abb5227ff1f1c/merged major:0 minor:162 fsType:overlay blockSize:0} overlay_0-167:{mountpoint:/var/lib/containers/storage/overlay/503d3b17c532a22eb9a733e57f94d4a24586dd56be22ee2490c1924b7b17efce/merged major:0 minor:167 fsType:overlay blockSize:0} overlay_0-173:{mountpoint:/var/lib/containers/storage/overlay/04d823890e17457d8b9938cd6c5c4b1a3419b40d46e44ffba959200cd6e66fd8/merged major:0 minor:173 fsType:overlay blockSize:0} overlay_0-175:{mountpoint:/var/lib/containers/storage/overlay/88cc443213a736c14a0b7940677d8e454c5622646f7703475743c5a1faf23014/merged major:0 minor:175 fsType:overlay blockSize:0} overlay_0-178:{mountpoint:/var/lib/containers/storage/overlay/66d554a7d99b922ce65ac99a897c2998e4a3d03886e006e4bc25556ba4997d22/merged major:0 minor:178 fsType:overlay blockSize:0} overlay_0-187:{mountpoint:/var/lib/containers/storage/overlay/3be7c3ce857258728859d4b1b41d11b8fb2aa108a52d7d26feae7fc5ca3bd8d6/merged major:0 minor:187 fsType:overlay blockSize:0} overlay_0-192:{mountpoint:/var/lib/containers/storage/overlay/9e40397c1d3cefb17ef6c9558a82ea7a4bef0c5b0b2f0a52d00662b45a6f80cf/merged major:0 minor:192 fsType:overlay blockSize:0} overlay_0-204:{mountpoint:/var/lib/containers/storage/overlay/447d5fff34a5b60f523b65e019b22c92fb1c6e5bd10b5776cfa5a4f18a250f8f/merged major:0 minor:204 fsType:overlay blockSize:0} overlay_0-212:{mountpoint:/var/lib/containers/storage/overlay/9ee42ea6cb1266ae40f971e7ca6bcb6fe7ce52e6f98337f1b675c921097a7dd1/merged major:0 minor:212 fsType:overlay blockSize:0} overlay_0-214:{mountpoint:/var/lib/containers/storage/overlay/30cb0b8a0767e9023ac34bc56ff66b56e56e7a563a339666f2efcb1aebb87f8d/merged major:0 minor:214 fsType:overlay blockSize:0} overlay_0-216:{mountpoint:/var/lib/containers/storage/overlay/8b96d773c52ebe4ef4434eff40f2bbde012fc07be55ef560e6180ce348b669fe/merged major:0 minor:216 fsType:overlay blockSize:0} overlay_0-218:{mountpoint:/var/lib/containers/storage/overlay/81e4d9f23318fa7a2fd35337ffc32ea5ccc2eeacc15af10f6cc68f769d4d1fde/merged major:0 minor:218 fsType:overlay blockSize:0} overlay_0-232:{mountpoint:/var/lib/containers/storage/overlay/ecc0e4a23f0552c8414cd1a698590a46a4b287bf7da065da3ddb2dd576f19047/merged major:0 minor:232 fsType:overlay blockSize:0} overlay_0-234:{mountpoint:/var/lib/containers/storage/overlay/399a645ac398e6e568698ac89a5b868b02c3981ffea9389d1a52e153a18984d0/merged major:0 minor:234 fsType:overlay blockSize:0} overlay_0-70:{mountpoint:/var/lib/containers/storage/overlay/7f644150e507d42d4d15a93178204b3beb8db50fdfcc15638bd24bb855fd9d58/merged major:0 minor:70 fsType:overlay blockSize:0} overlay_0-72:{mountpoint:/var/lib/containers/storage/overlay/355c12e7a2a26c269a4ecac2cd94d08743a9164377f741ff6a19d8372d3b7780/merged major:0 minor:72 fsType:overlay blockSize:0} overlay_0-74:{mountpoint:/var/lib/containers/storage/overlay/34343e939993684bb99562e7fb974f0cb752b260d7c320cab48f76de86737ce4/merged major:0 minor:74 fsType:overlay blockSize:0} overlay_0-76:{mountpoint:/var/lib/containers/storage/overlay/dbe8071b67cae6528b0e71f4afddb7a247d3522fc5c534acf42418b4abff7bf8/merged major:0 minor:76 fsType:overlay blockSize:0} overlay_0-78:{mountpoint:/var/lib/containers/storage/overlay/b07eeea50ca2ee780a377ba2300177f017f755ef6dac85df2086c851fcc7b42c/merged major:0 minor:78 fsType:overlay blockSize:0} overlay_0-80:{mountpoint:/var/lib/containers/storage/overlay/5d6c8e649d9bb97f5d86407bdc9498bc1e2196d77fbf524ff3a59ecf8c92b9bf/merged major:0 minor:80 fsType:overlay blockSize:0} overlay_0-84:{mountpoint:/var/lib/containers/storage/overlay/e24a442901f839ff1156bb34207e497779131299d62b7f5868fb4c5d85581385/merged major:0 minor:84 fsType:overlay blockSize:0} overlay_0-88:{mountpoint:/var/lib/containers/storage/overlay/18f9110ed0fd6f09da1b401f8c05e2b6e819c66a0328ff37cc17e51eb5a8457a/merged major:0 minor:88 fsType:overlay blockSize:0} overlay_0-92:{mountpoint:/var/lib/containers/storage/overlay/764a6114a5ae33633bbd304d20541d64fb58f9809a09d27b8df689919960198b/merged major:0 minor:92 fsType:overlay blockSize:0} overlay_0-96:{mountpoint:/var/lib/containers/storage/overlay/17c13556df516f124fe1cc8113297e6622bd84608ac7fffd555850949796dd4e/merged major:0 minor:96 fsType:overlay blockSize:0}]
Oct 30 21:52:59.587640 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.568804    4412 manager.go:217] Machine: {Timestamp:2026-10-30 21:52:59.568349467 +0000 UTC m=+0.073021410 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3098719 MemoryCapacity:32812015616 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2b39ebd184acc12d51b4696614397b SystemUUID:ec2b39eb-d184-acc1-2d51-b4696614397b BootID:ce4b67e4-503b-437c-8dbe-13e77d47e3f4 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:24 Capacity:16406007808 Type:vfs Inodes:4005373 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/164bafd8-ff17-46a2-afc0-dd01fa1aaeb6/volumes/kubernetes.io~projected/kube-api-access-78t6j DeviceMajor:0 DeviceMinor:57 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-102 DeviceMajor:0 DeviceMinor:102 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-109 DeviceMajor:0 DeviceMinor:109 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-132 DeviceMajor:0 DeviceMinor:132 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-147 DeviceMajor:0 DeviceMinor:147 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/2345ff5b-db97-4855-8adb-59b9a780615e/volumes/kubernetes.io~projected/kube-api-access-c622t DeviceMajor:0 DeviceMinor:197 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/982efb96-3613-4ef2-ac51-ed3f40745a32/volumes/kubernetes.io~empty-dir/tmp DeviceMajor:0 DeviceMinor:49 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/b1a02001-9121-47bd-ae96-b39196ad2357/volumes/kubernetes.io~projected/kube-api-access-vsg28 DeviceMajor:0 DeviceMinor:55 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-218 DeviceMajor:0 DeviceMinor:218 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/982efb96-3613-4ef2-ac51-ed3f40745a32/volumes/kubernetes.io~empty-dir/etc-tuned DeviceMajor:0 DeviceMinor:47 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/bf0e36d0-87bd-4e1e-be40-8a3a0f856987/volumes/kubernetes.io~projected/kube-api-access-ctg2b DeviceMajor:0 DeviceMinor:52 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-119 DeviceMajor:0 DeviceMinor:119 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-149 DeviceMajor:0 DeviceMinor:149 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-167 DeviceMajor:0 DeviceMinor:167 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-204 DeviceMajor:0 DeviceMinor:204 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-232 DeviceMajor:0 DeviceMinor:232 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:32 Capacity:16406007808 Type:vfs Inodes:1048576 HasInodes:true} {Device:overlay_0-100 DeviceMajor:0 DeviceMinor:100 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/7e6b0df0-07eb-4301-adaa-1a6514639d9c/volumes/kubernetes.io~projected/kube-api-access-fsf4l DeviceMajor:0 DeviceMinor:229 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/bf0e36d0-87bd-4e1e-be40-8a3a0f856987/volumes/kubernetes.io~secret/ovn-node-metrics-cert DeviceMajor:0 DeviceMinor:48 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/c590d5fe-57de-4905-90fe-8527b215ce94/volumes/kubernetes.io~projected/kube-api-access-fq868 DeviceMajor:0 DeviceMinor:51 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/9751c27f53d33b0e6f09e8d175690fc83db3fe0e1f08314102e30aca663fe0c9/userdata/shm DeviceMajor:0 DeviceMinor:62 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-96 DeviceMajor:0 DeviceMinor:96 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/c8ab0219dae8058a1a0ad4e07bca8ce1cc695a209049fe6f3e048c5e56ae3da0/userdata/shm DeviceMajor:0 DeviceMinor:202 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/3f3feb24d73c361bf3c8b6abc3c8ab7edc3aa0f9549ddbec02315f3dd7ed0b0c/userdata/shm DeviceMajor:0 DeviceMinor:210 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-216 DeviceMajor:0 DeviceMinor:216 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/25590e1f-99e3-4469-8a69-39cd30317c28/volumes/kubernetes.io~secret/agent-certs DeviceMajor:0 DeviceMinor:50 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/386f9569-a5f5-4a9e-beee-3005994186cd/volumes/kubernetes.io~projected/kube-api-access-tg4f9 DeviceMajor:0 DeviceMinor:59 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-74 DeviceMajor:0 DeviceMinor:74 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-76 DeviceMajor:0 DeviceMinor:76 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-88 DeviceMajor:0 DeviceMinor:88 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/705914d7dce2552b00ce3ab62bd974e25578a0cc87f776136f313b6d4f04b10e/userdata/shm DeviceMajor:0 DeviceMinor:90 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/7be00c31-281f-4d1c-8e12-1bf74bbe7cbd/volumes/kubernetes.io~projected/kube-api-access-6j99v DeviceMajor:0 DeviceMinor:207 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/dffd7a70-b7f3-4e1b-b3ab-74c5a574cab2/volumes/kubernetes.io~projected/kube-api-access-hrlj5 DeviceMajor:0 DeviceMinor:56 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-72 DeviceMajor:0 DeviceMinor:72 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-139 DeviceMajor:0 DeviceMinor:139 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-173 DeviceMajor:0 DeviceMinor:173 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-212 DeviceMajor:0 DeviceMinor:212 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/5a0d9b46d476286e5b9f13aa9ed77125183273c6d32c4e1fb87b7608b8e29b35/userdata/shm DeviceMajor:0 DeviceMinor:230 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/6e596f1f222be642b03bca48eb0148c607deb4bab720ed68a09e44e31f2c0dea/userdata/shm DeviceMajor:0 DeviceMinor:86 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-137 DeviceMajor:0 DeviceMinor:137 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/db073c3b-7cee-4e22-8c11-209bbeb21a96/volumes/kubernetes.io~projected/kube-api-access-vd7d5 DeviceMajor:0 DeviceMinor:224 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/982efb96-3613-4ef2-ac51-ed3f40745a32/volumes/kubernetes.io~projected/kube-api-access-2pwzh DeviceMajor:0 DeviceMinor:53 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-84 DeviceMajor:0 DeviceMinor:84 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/9e1ebfcd253a8ad808556ec9db0d8260ed7f2447b3ad5380483f2d6e13239593/userdata/shm DeviceMajor:0 DeviceMinor:94 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-104 DeviceMajor:0 DeviceMinor:104 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-157 DeviceMajor:0 DeviceMinor:157 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-187 DeviceMajor:0 DeviceMinor:187 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-214 DeviceMajor:0 DeviceMinor:214 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/693c46ffe0fe055558dabec72ff633a79b91a6e9ec57f3e27c1c31178984aaa7/userdata/shm DeviceMajor:0 DeviceMinor:60 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-78 DeviceMajor:0 DeviceMinor:78 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/bf0e36d0-87bd-4e1e-be40-8a3a0f856987/volume-subpaths/run-systemd/ovnkube-controller/6 DeviceMajor:0 DeviceMinor:26 Capacity:6562406400 Type:vfs Inodes:819200 HasInodes:true} {Device:overlay_0-192 DeviceMajor:0 DeviceMinor:192 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/d17b7754-d1d4-4952-97c7-7a04c714b48c/volumes/kubernetes.io~secret/original-pull-secret DeviceMajor:0 DeviceMinor:206 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/eecea248f97ac1f5faa4ef08a1b96061cee2cce9cac0767037b2d544ff1a8c1d/userdata/shm DeviceMajor:0 DeviceMinor:208 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/da317ccc84703e328e24540d39e76dcf76ce091d7baad369756dba3796160cfb/userdata/shm DeviceMajor:0 DeviceMinor:64 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-92 DeviceMajor:0 DeviceMinor:92 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-178 DeviceMajor:0 DeviceMinor:178 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-175 DeviceMajor:0 DeviceMinor:175 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/a4ff4cce-671b-4136-8b63-b22148c0db4e/volumes/kubernetes.io~projected/kube-api-access-6r8d9 DeviceMajor:0 DeviceMinor:54 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/a9887498d2723b7a9e696b07db4f758f0c7f19897e1b590ceb5739927bce13cb/userdata/shm DeviceMajor:0 DeviceMinor:61 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/437d7c0fb64e2862943d48eb6c7731f8d740ca3387b8624fad4e28e5b29e1511/userdata/shm DeviceMajor:0 DeviceMinor:63 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-70 DeviceMajor:0 DeviceMinor:70 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/3970e1be64d4877f260bff03ca6a43749580fdc4c5a49479929398c86576395e/userdata/shm DeviceMajor:0 DeviceMinor:98 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-124 DeviceMajor:0 DeviceMinor:124 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-234 DeviceMajor:0 DeviceMinor:234 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/8806f1e1-0057-4738-8fbc-7e503e1f91ca/volumes/kubernetes.io~projected/kube-api-access-fwsxm DeviceMajor:0 DeviceMinor:58 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/3f2b3c7943e258b620f38697047406863aa88efda8d5faada68af4c4e955aeb5/userdata/shm DeviceMajor:0 DeviceMinor:82 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:composefs_0-34 DeviceMajor:0 DeviceMinor:34 Capacity:6422528 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:overlay_0-80 DeviceMajor:0 DeviceMinor:80 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-111 DeviceMajor:0 DeviceMinor:111 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-127 DeviceMajor:0 DeviceMinor:127 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:26 Capacity:6562406400 Type:vfs Inodes:819200 HasInodes:true} {Device:/var/lib/kubelet/pods/2345ff5b-db97-4855-8adb-59b9a780615e/volumes/kubernetes.io~secret/cert DeviceMajor:0 DeviceMinor:45 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/4a03a836-506f-404e-9c55-c26d779cd4ee/volumes/kubernetes.io~projected/kube-api-access-dtsnb DeviceMajor:0 DeviceMinor:201 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:overlay_0-162 DeviceMajor:0 DeviceMinor:162 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:3f3feb24d73c361 MacAddress:12:8c:4f:5a:35:d6 Speed:10000 Mtu:8901} {Name:5a0d9b46d476286 MacAddress:3a:ae:1f:17:40:eb Speed:10000 Mtu:8901} {Name:br-ex MacAddress:02:48:66:66:c1:e7 Speed:0 Mtu:9001} {Name:br-int MacAddress:ba:1c:d4:6a:5a:0b Speed:0 Mtu:8901} {Name:c8ab0219dae8058 MacAddress:56:e9:6c:7e:4a:09 Speed:10000 Mtu:8901} {Name:eecea248f97ac1f MacAddress:86:46:cd:99:14:07 Speed:10000 Mtu:8901} {Name:ens5 MacAddress:02:48:66:66:c1:e7 Speed:0 Mtu:9001} {Name:genev_sys_6081 MacAddress:42:08:fc:9b:67:96 Speed:0 Mtu:65000} {Name:ovn-k8s-mp0 MacAddress:0a:58:0a:85:00:02 Speed:0 Mtu:8901} {Name:ovs-system MacAddress:ee:6b:d9:0c:87:57 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812015616 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Oct 30 21:52:59.587640 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.569039    4412 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Oct 30 21:52:59.587640 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.569078    4412 manager.go:233] Version: {KernelVersion:5.14.0-687.60.1.el9_8.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.8.20261023-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Oct 30 21:52:59.587640 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.569353    4412 container_manager_linux.go:275] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Oct 30 21:52:59.587640 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.569371    4412 container_manager_linux.go:280] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-131-202.us-west-1.compute.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"/system.slice","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{},"system-reserved-compressible":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Oct 30 21:52:59.587640 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.569498    4412 topology_manager.go:143] "Creating topology manager with none policy"
Oct 30 21:52:59.587640 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.569506    4412 container_manager_linux.go:311] "Creating device plugin manager"
Oct 30 21:52:59.587640 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.569518    4412 manager.go:144] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Oct 30 21:52:59.587640 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.569531    4412 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Oct 30 21:52:59.587640 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.569609    4412 container_manager_linux.go:320] "Creating Dynamic Resource Allocation (DRA) manager"
Oct 30 21:52:59.587640 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.569797    4412 state_mem.go:41] "Initialized" logger="CPUManager state memory"
Oct 30 21:52:59.587640 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.569921    4412 server.go:1235] "Using root directory" path="/var/lib/kubelet"
Oct 30 21:52:59.587640 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.569954    4412 kubelet.go:493] "Attempting to sync node with API server"
Oct 30 21:52:59.587640 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.569964    4412 kubelet.go:394] "Adding static pod path" path="/etc/kubernetes/manifests"
Oct 30 21:52:59.587640 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.569973    4412 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Oct 30 21:52:59.587640 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.569981    4412 kubelet.go:405] "Adding apiserver pod source"
Oct 30 21:52:59.587640 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.569989    4412 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Oct 30 21:52:59.587640 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.570604    4412 state_checkpoint.go:81] "State checkpoint: restored pod resource state from checkpoint"
Oct 30 21:52:59.587640 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.570614    4412 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Oct 30 21:52:59.587640 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.571060    4412 kuberuntime_manager.go:304] "Container runtime initialized" containerRuntime="cri-o" version="1.35.10-3.rhaos4.22.git9383502.el9" apiVersion="v1"
Oct 30 21:52:59.587640 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.578465    4412 state_checkpoint.go:81] "State checkpoint: restored pod resource state from checkpoint"
Oct 30 21:52:59.587640 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.578474    4412 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Oct 30 21:52:59.587640 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.578573    4412 certificate_store.go:147] "Loading cert/key pair from a file" filePath="/var/lib/kubelet/pki/kubelet-server-current.pem"
Oct 30 21:52:59.587640 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.578749    4412 kubelet.go:963] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Oct 30 21:52:59.587640 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.578775    4412 kubelet.go:990] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Oct 30 21:52:59.587640 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.578861    4412 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Oct 30 21:52:59.587640 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.578873    4412 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Oct 30 21:52:59.587640 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.578880    4412 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Oct 30 21:52:59.587640 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.578889    4412 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Oct 30 21:52:59.587640 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.578898    4412 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Oct 30 21:52:59.587640 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.578905    4412 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Oct 30 21:52:59.587640 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.578910    4412 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Oct 30 21:52:59.587640 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.578916    4412 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Oct 30 21:52:59.587640 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.578922    4412 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Oct 30 21:52:59.587640 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.578928    4412 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Oct 30 21:52:59.587640 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.578936    4412 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Oct 30 21:52:59.587640 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.578949    4412 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Oct 30 21:52:59.587640 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.578977    4412 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Oct 30 21:52:59.587640 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.578985    4412 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Oct 30 21:52:59.587640 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.582343    4412 server.go:1263] "Started kubelet"
Oct 30 21:52:59.587640 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.582441    4412 server.go:182] "Starting to listen" address="0.0.0.0" port=10250
Oct 30 21:52:59.587640 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.582431    4412 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Oct 30 21:52:59.587640 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.582504    4412 server_v1.go:49] "podresources" method="list" useActivePods=true
Oct 30 21:52:59.587640 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.582892    4412 server.go:254] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Oct 30 21:52:59.587640 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.583519    4412 server.go:317] "Adding debug handlers to kubelet server"
Oct 30 21:52:59.587640 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.585238    4412 fs_resource_analyzer.go:69] "Starting FS ResourceAnalyzer"
Oct 30 21:52:59.587640 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.585268    4412 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Oct 30 21:52:59.587640 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.585317    4412 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-10-29 21:47:21 +0000 UTC" deadline="2028-04-24 12:32:08.828553165 +0000 UTC"
Oct 30 21:52:59.587640 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.585347    4412 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12998h39m9.243209298s"
Oct 30 21:52:59.587640 ip-10-0-131-202 kubenswrapper[4412]: E1030 21:52:59.585409    4412 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-131-202.us-west-1.compute.internal\" not found"
Oct 30 21:52:59.587640 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.585555    4412 volume_manager.go:309] "The desired_state_of_world populator starts"
Oct 30 21:52:59.587640 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.585565    4412 volume_manager.go:311] "Starting Kubelet Volume Manager"
Oct 30 21:52:59.587640 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.585594    4412 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Oct 30 21:52:59.587640 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.587529    4412 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="c590d5fe-57de-4905-90fe-8527b215ce94" volumeName="kubernetes.io/configmap/c590d5fe-57de-4905-90fe-8527b215ce94-cni-binary-copy" seLinuxMountContext=""
Oct 30 21:52:59.587640 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.587565    4412 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="8806f1e1-0057-4738-8fbc-7e503e1f91ca" volumeName="kubernetes.io/projected/8806f1e1-0057-4738-8fbc-7e503e1f91ca-kube-api-access-fwsxm" seLinuxMountContext=""
Oct 30 21:52:59.587640 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.587580    4412 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="c590d5fe-57de-4905-90fe-8527b215ce94" volumeName="kubernetes.io/projected/c590d5fe-57de-4905-90fe-8527b215ce94-kube-api-access-fq868" seLinuxMountContext=""
Oct 30 21:52:59.587640 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.587605    4412 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d17b7754-d1d4-4952-97c7-7a04c714b48c" volumeName="kubernetes.io/secret/d17b7754-d1d4-4952-97c7-7a04c714b48c-original-pull-secret" seLinuxMountContext=""
Oct 30 21:52:59.587640 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.587631    4412 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="25590e1f-99e3-4469-8a69-39cd30317c28" volumeName="kubernetes.io/configmap/25590e1f-99e3-4469-8a69-39cd30317c28-konnectivity-ca" seLinuxMountContext=""
Oct 30 21:52:59.587640 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.587656    4412 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7be00c31-281f-4d1c-8e12-1bf74bbe7cbd" volumeName="kubernetes.io/projected/7be00c31-281f-4d1c-8e12-1bf74bbe7cbd-kube-api-access-6j99v" seLinuxMountContext=""
Oct 30 21:52:59.587640 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.587668    4412 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="982efb96-3613-4ef2-ac51-ed3f40745a32" volumeName="kubernetes.io/empty-dir/982efb96-3613-4ef2-ac51-ed3f40745a32-tmp" seLinuxMountContext=""
Oct 30 21:52:59.590376 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.587681    4412 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a4ff4cce-671b-4136-8b63-b22148c0db4e" volumeName="kubernetes.io/projected/a4ff4cce-671b-4136-8b63-b22148c0db4e-kube-api-access-6r8d9" seLinuxMountContext=""
Oct 30 21:52:59.590376 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.587695    4412 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="bf0e36d0-87bd-4e1e-be40-8a3a0f856987" volumeName="kubernetes.io/configmap/bf0e36d0-87bd-4e1e-be40-8a3a0f856987-env-overrides" seLinuxMountContext=""
Oct 30 21:52:59.590376 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.587717    4412 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="bf0e36d0-87bd-4e1e-be40-8a3a0f856987" volumeName="kubernetes.io/configmap/bf0e36d0-87bd-4e1e-be40-8a3a0f856987-ovnkube-script-lib" seLinuxMountContext=""
Oct 30 21:52:59.590376 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.587731    4412 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="25590e1f-99e3-4469-8a69-39cd30317c28" volumeName="kubernetes.io/secret/25590e1f-99e3-4469-8a69-39cd30317c28-agent-certs" seLinuxMountContext=""
Oct 30 21:52:59.590376 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.587758    4412 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="982efb96-3613-4ef2-ac51-ed3f40745a32" volumeName="kubernetes.io/projected/982efb96-3613-4ef2-ac51-ed3f40745a32-kube-api-access-2pwzh" seLinuxMountContext=""
Oct 30 21:52:59.590376 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.587771    4412 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="c590d5fe-57de-4905-90fe-8527b215ce94" volumeName="kubernetes.io/configmap/c590d5fe-57de-4905-90fe-8527b215ce94-whereabouts-flatfile-configmap" seLinuxMountContext=""
Oct 30 21:52:59.590376 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.587782    4412 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="db073c3b-7cee-4e22-8c11-209bbeb21a96" volumeName="kubernetes.io/empty-dir/db073c3b-7cee-4e22-8c11-209bbeb21a96-data-volume" seLinuxMountContext=""
Oct 30 21:52:59.590376 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.587807    4412 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="386f9569-a5f5-4a9e-beee-3005994186cd" volumeName="kubernetes.io/projected/386f9569-a5f5-4a9e-beee-3005994186cd-kube-api-access-tg4f9" seLinuxMountContext=""
Oct 30 21:52:59.590376 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.587848    4412 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4a03a836-506f-404e-9c55-c26d779cd4ee" volumeName="kubernetes.io/configmap/4a03a836-506f-404e-9c55-c26d779cd4ee-config-volume" seLinuxMountContext=""
Oct 30 21:52:59.590376 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.587860    4412 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b1a02001-9121-47bd-ae96-b39196ad2357" volumeName="kubernetes.io/projected/b1a02001-9121-47bd-ae96-b39196ad2357-kube-api-access-vsg28" seLinuxMountContext=""
Oct 30 21:52:59.590376 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.587882    4412 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="164bafd8-ff17-46a2-afc0-dd01fa1aaeb6" volumeName="kubernetes.io/configmap/164bafd8-ff17-46a2-afc0-dd01fa1aaeb6-serviceca" seLinuxMountContext=""
Oct 30 21:52:59.590376 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.587895    4412 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="2345ff5b-db97-4855-8adb-59b9a780615e" volumeName="kubernetes.io/projected/2345ff5b-db97-4855-8adb-59b9a780615e-kube-api-access-c622t" seLinuxMountContext=""
Oct 30 21:52:59.590376 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.587906    4412 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="2345ff5b-db97-4855-8adb-59b9a780615e" volumeName="kubernetes.io/secret/2345ff5b-db97-4855-8adb-59b9a780615e-cert" seLinuxMountContext=""
Oct 30 21:52:59.590376 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.587927    4412 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b1a02001-9121-47bd-ae96-b39196ad2357" volumeName="kubernetes.io/configmap/b1a02001-9121-47bd-ae96-b39196ad2357-cni-binary-copy" seLinuxMountContext=""
Oct 30 21:52:59.590376 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.587949    4412 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b1a02001-9121-47bd-ae96-b39196ad2357" volumeName="kubernetes.io/configmap/b1a02001-9121-47bd-ae96-b39196ad2357-multus-daemon-config" seLinuxMountContext=""
Oct 30 21:52:59.590376 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.587959    4412 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="bf0e36d0-87bd-4e1e-be40-8a3a0f856987" volumeName="kubernetes.io/configmap/bf0e36d0-87bd-4e1e-be40-8a3a0f856987-ovnkube-config" seLinuxMountContext=""
Oct 30 21:52:59.590376 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.587978    4412 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="bf0e36d0-87bd-4e1e-be40-8a3a0f856987" volumeName="kubernetes.io/secret/bf0e36d0-87bd-4e1e-be40-8a3a0f856987-ovn-node-metrics-cert" seLinuxMountContext=""
Oct 30 21:52:59.590376 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.587998    4412 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="c590d5fe-57de-4905-90fe-8527b215ce94" volumeName="kubernetes.io/configmap/c590d5fe-57de-4905-90fe-8527b215ce94-cni-sysctl-allowlist" seLinuxMountContext=""
Oct 30 21:52:59.590376 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.588010    4412 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4a03a836-506f-404e-9c55-c26d779cd4ee" volumeName="kubernetes.io/empty-dir/4a03a836-506f-404e-9c55-c26d779cd4ee-tmp-dir" seLinuxMountContext=""
Oct 30 21:52:59.590376 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.588022    4412 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="bf0e36d0-87bd-4e1e-be40-8a3a0f856987" volumeName="kubernetes.io/projected/bf0e36d0-87bd-4e1e-be40-8a3a0f856987-kube-api-access-ctg2b" seLinuxMountContext=""
Oct 30 21:52:59.590376 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.588048    4412 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4a03a836-506f-404e-9c55-c26d779cd4ee" volumeName="kubernetes.io/projected/4a03a836-506f-404e-9c55-c26d779cd4ee-kube-api-access-dtsnb" seLinuxMountContext=""
Oct 30 21:52:59.590376 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.588087    4412 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="982efb96-3613-4ef2-ac51-ed3f40745a32" volumeName="kubernetes.io/empty-dir/982efb96-3613-4ef2-ac51-ed3f40745a32-etc-tuned" seLinuxMountContext=""
Oct 30 21:52:59.590376 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.588106    4412 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="db073c3b-7cee-4e22-8c11-209bbeb21a96" volumeName="kubernetes.io/projected/db073c3b-7cee-4e22-8c11-209bbeb21a96-kube-api-access-vd7d5" seLinuxMountContext=""
Oct 30 21:52:59.590376 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.588143    4412 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="dffd7a70-b7f3-4e1b-b3ab-74c5a574cab2" volumeName="kubernetes.io/empty-dir/dffd7a70-b7f3-4e1b-b3ab-74c5a574cab2-tmp-dir" seLinuxMountContext=""
Oct 30 21:52:59.590376 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.588154    4412 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="dffd7a70-b7f3-4e1b-b3ab-74c5a574cab2" volumeName="kubernetes.io/projected/dffd7a70-b7f3-4e1b-b3ab-74c5a574cab2-kube-api-access-hrlj5" seLinuxMountContext=""
Oct 30 21:52:59.590376 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.588167    4412 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="164bafd8-ff17-46a2-afc0-dd01fa1aaeb6" volumeName="kubernetes.io/projected/164bafd8-ff17-46a2-afc0-dd01fa1aaeb6-kube-api-access-78t6j" seLinuxMountContext=""
Oct 30 21:52:59.590376 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.588185    4412 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7e6b0df0-07eb-4301-adaa-1a6514639d9c" volumeName="kubernetes.io/projected/7e6b0df0-07eb-4301-adaa-1a6514639d9c-kube-api-access-fsf4l" seLinuxMountContext=""
Oct 30 21:52:59.590376 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.588197    4412 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a4ff4cce-671b-4136-8b63-b22148c0db4e" volumeName="kubernetes.io/configmap/a4ff4cce-671b-4136-8b63-b22148c0db4e-iptables-alerter-script" seLinuxMountContext=""
Oct 30 21:52:59.590376 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.588216    4412 reconstruct.go:97] "Volume reconstruction finished"
Oct 30 21:52:59.590376 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.588223    4412 reconciler.go:29] "Reconciler: start to sync state"
Oct 30 21:52:59.590376 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.588345    4412 factory.go:55] Registering systemd factory
Oct 30 21:52:59.590376 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.588363    4412 factory.go:223] Registration of the systemd container factory successfully
Oct 30 21:52:59.590376 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.588732    4412 factory.go:153] Registering CRI-O factory
Oct 30 21:52:59.590376 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.588747    4412 factory.go:223] Registration of the crio container factory successfully
Oct 30 21:52:59.590376 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.588921    4412 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Oct 30 21:52:59.590376 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.588968    4412 factory.go:103] Registering Raw factory
Oct 30 21:52:59.590376 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.588986    4412 manager.go:1199] Started watching for new ooms in manager
Oct 30 21:52:59.590376 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.590136    4412 manager.go:319] Starting recovery of all containers
Oct 30 21:52:59.619414 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.619382    4412 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Oct 30 21:52:59.620747 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.620722    4412 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Oct 30 21:52:59.620864 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.620776    4412 status_manager.go:249] "Starting to sync pod status with apiserver"
Oct 30 21:52:59.620864 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.620801    4412 kubelet.go:2521] "Starting kubelet main sync loop"
Oct 30 21:52:59.620864 ip-10-0-131-202 kubenswrapper[4412]: E1030 21:52:59.620841    4412 kubelet.go:2545] "Skipping pod synchronization" err="[container runtime status check may not have completed yet, PLEG is not healthy: pleg has yet to be successful]"
Oct 30 21:52:59.628797 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.628760    4412 generic.go:356] "Generic (PLEG): container finished" podID="c590d5fe-57de-4905-90fe-8527b215ce94" containerID="bb3e7be2868bb3748f42810e1a854f57d1614e6e98a15ea8282ec5d345c83c85" exitCode=0
Oct 30 21:52:59.628797 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.628797    4412 generic.go:356] "Generic (PLEG): container finished" podID="c590d5fe-57de-4905-90fe-8527b215ce94" containerID="0d21058138ab1ee3fa62a5259ef5c21d8e805bf8a90a24ded701fa3f4af95bca" exitCode=0
Oct 30 21:52:59.628937 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.628803    4412 generic.go:356] "Generic (PLEG): container finished" podID="c590d5fe-57de-4905-90fe-8527b215ce94" containerID="0f9580bee00ff200dd76780b6aaeb6ae57fa603d544e06a7d591e53f6ce07cfe" exitCode=0
Oct 30 21:52:59.628937 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.628814    4412 generic.go:356] "Generic (PLEG): container finished" podID="c590d5fe-57de-4905-90fe-8527b215ce94" containerID="c17bf4a32f01280e4ed086fdb68fa78e9a0e961dd25f50f6c5be54d57c1e0a20" exitCode=0
Oct 30 21:52:59.628937 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.628821    4412 generic.go:356] "Generic (PLEG): container finished" podID="c590d5fe-57de-4905-90fe-8527b215ce94" containerID="ac58b695732aac4d4740d32ea93510c8c01d7aa4c2c7a97e262a4de55cec5f97" exitCode=0
Oct 30 21:52:59.628937 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.628827    4412 generic.go:356] "Generic (PLEG): container finished" podID="c590d5fe-57de-4905-90fe-8527b215ce94" containerID="1b3b49caae15fa36b2e5d6780f25b557570358db3a35a3f1bec2561271156924" exitCode=0
Oct 30 21:52:59.634590 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.634566    4412 generic.go:356] "Generic (PLEG): container finished" podID="ec1e59c1a0a508a7c75743ab25eff8ba" containerID="25993f737081014a0fe5f6981adc9bdc77e6a44d7aab47aa22ecfffbf9c40ee5" exitCode=0
Oct 30 21:52:59.654744 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.654726    4412 manager.go:324] Recovery completed
Oct 30 21:52:59.668412 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.668392    4412 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 21:52:59.669806 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.669789    4412 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-131-202.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Oct 30 21:52:59.669896 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.669824    4412 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-131-202.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 21:52:59.669896 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.669838    4412 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-131-202.us-west-1.compute.internal" event="NodeHasSufficientPID"
Oct 30 21:52:59.671075 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.671060    4412 cpu_manager.go:226] "Starting" policy="none"
Oct 30 21:52:59.671075 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.671072    4412 cpu_manager.go:227] "Reconciling" reconcilePeriod="10s"
Oct 30 21:52:59.671206 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.671077    4412 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 21:52:59.671206 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.671091    4412 state_mem.go:41] "Initialized" logger="CPUManager state checkpoint.CPUManager state memory"
Oct 30 21:52:59.671315 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.671250    4412 state_mem.go:94] "Updated default CPUSet" logger="CPUManager state checkpoint.CPUManager state memory" cpuSet=""
Oct 30 21:52:59.671315 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.671266    4412 state_mem.go:102] "Updated CPUSet assignments" logger="CPUManager state checkpoint.CPUManager state memory" assignments={}
Oct 30 21:52:59.671315 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.671291    4412 state_checkpoint.go:142] "restored state from checkpoint" logger="CPUManager state checkpoint"
Oct 30 21:52:59.671315 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.671299    4412 state_checkpoint.go:143] "defaultCPUSet" logger="CPUManager state checkpoint" defaultCpuSet=""
Oct 30 21:52:59.671315 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.671308    4412 policy_none.go:50] "Start"
Oct 30 21:52:59.671528 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.671320    4412 memory_manager.go:187] "Starting memorymanager" policy="None"
Oct 30 21:52:59.671528 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.671333    4412 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Oct 30 21:52:59.671624 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.671574    4412 state_mem.go:77] "Updated machine memory state" logger="Memory Manager state checkpoint"
Oct 30 21:52:59.671624 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.671585    4412 state_checkpoint.go:85] "State checkpoint: restored state from checkpoint" logger="Memory Manager state checkpoint"
Oct 30 21:52:59.671624 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.671593    4412 policy_none.go:44] "Start"
Oct 30 21:52:59.671903 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.671888    4412 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-131-202.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Oct 30 21:52:59.671968 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.671920    4412 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-131-202.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 21:52:59.671968 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.671931    4412 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-131-202.us-west-1.compute.internal" event="NodeHasSufficientPID"
Oct 30 21:52:59.675470 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.675446    4412 node_container_manager_linux.go:130] "Enforcing system reserved compressible on cgroup" cgroupName="/system.slice" limits={"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"}
Oct 30 21:52:59.679169 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.679154    4412 manager.go:341] "Starting Device Plugin manager"
Oct 30 21:52:59.679264 ip-10-0-131-202 kubenswrapper[4412]: E1030 21:52:59.679202    4412 manager.go:525] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Oct 30 21:52:59.679264 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.679212    4412 server.go:87] "Starting device plugin registration server"
Oct 30 21:52:59.679451 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.679439    4412 eviction_manager.go:194] "Eviction manager: starting control loop"
Oct 30 21:52:59.679495 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.679451    4412 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Oct 30 21:52:59.680110 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.679569    4412 plugin_watcher.go:53] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Oct 30 21:52:59.680110 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.679670    4412 plugin_watcher.go:201] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Oct 30 21:52:59.680110 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.679709    4412 plugin_manager.go:119] "The desired_state_of_world populator (plugin watcher) starts"
Oct 30 21:52:59.680110 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.679716    4412 plugin_manager.go:121] "Starting Kubelet Plugin Manager"
Oct 30 21:52:59.680110 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.679749    4412 reconciler.go:167] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-10-30T21:52:59.679698862Z","UUID":"05eb2a06-20ac-4ed0-a52a-4b74d658ff64","Handler":null,"Name":"","Endpoint":""}
Oct 30 21:52:59.680800 ip-10-0-131-202 kubenswrapper[4412]: E1030 21:52:59.680779    4412 eviction_manager.go:272] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Oct 30 21:52:59.680911 ip-10-0-131-202 kubenswrapper[4412]: E1030 21:52:59.680818    4412 eviction_manager.go:297] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-131-202.us-west-1.compute.internal\" not found"
Oct 30 21:52:59.681213 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.681193    4412 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Oct 30 21:52:59.681213 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.681215    4412 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Oct 30 21:52:59.721262 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.721225    4412 kubelet.go:2608] "SyncLoop ADD" source="file" pods=["kube-system/kube-apiserver-proxy-ip-10-0-131-202.us-west-1.compute.internal","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-202.us-west-1.compute.internal"]
Oct 30 21:52:59.721347 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.721336    4412 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 21:52:59.722198 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.722179    4412 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-131-202.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Oct 30 21:52:59.722280 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.722217    4412 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-131-202.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 21:52:59.722280 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.722231    4412 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-131-202.us-west-1.compute.internal" event="NodeHasSufficientPID"
Oct 30 21:52:59.722505 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.722492    4412 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 21:52:59.722631 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.722611    4412 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 21:52:59.723273 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.723254    4412 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-131-202.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Oct 30 21:52:59.723352 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.723284    4412 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-131-202.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 21:52:59.723352 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.723295    4412 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-131-202.us-west-1.compute.internal" event="NodeHasSufficientPID"
Oct 30 21:52:59.723352 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.723259    4412 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-131-202.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Oct 30 21:52:59.723494 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.723355    4412 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-131-202.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 21:52:59.723494 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.723367    4412 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-131-202.us-west-1.compute.internal" event="NodeHasSufficientPID"
Oct 30 21:52:59.723494 ip-10-0-131-202 kubenswrapper[4412]: E1030 21:52:59.723413    4412 kubelet.go:3364] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-131-202.us-west-1.compute.internal\" not found" node="ip-10-0-131-202.us-west-1.compute.internal"
Oct 30 21:52:59.723611 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.723583    4412 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-202.us-west-1.compute.internal" event={"ID":"ec1e59c1a0a508a7c75743ab25eff8ba","Type":"ContainerStarted","Data":"d337aed051dcee497300943e642bba3c14228bb4e06427a412d286f8e500ae25"}
Oct 30 21:52:59.723662 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.723616    4412 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-202.us-west-1.compute.internal" event={"ID":"ec1e59c1a0a508a7c75743ab25eff8ba","Type":"ContainerDied","Data":"25993f737081014a0fe5f6981adc9bdc77e6a44d7aab47aa22ecfffbf9c40ee5"}
Oct 30 21:52:59.723662 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.723630    4412 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-202.us-west-1.compute.internal" event={"ID":"ec1e59c1a0a508a7c75743ab25eff8ba","Type":"ContainerStarted","Data":"9751c27f53d33b0e6f09e8d175690fc83db3fe0e1f08314102e30aca663fe0c9"}
Oct 30 21:52:59.723662 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.723659    4412 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-202.us-west-1.compute.internal" event={"ID":"04337a6f3666f0af07c3d2b702c6e379","Type":"ContainerStarted","Data":"ffb6e25f287382f2c628231ad8a3ad3fed2eaba2281db36a37cdce8064bebbc6"}
Oct 30 21:52:59.723801 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.723671    4412 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-202.us-west-1.compute.internal" event={"ID":"04337a6f3666f0af07c3d2b702c6e379","Type":"ContainerStarted","Data":"693c46ffe0fe055558dabec72ff633a79b91a6e9ec57f3e27c1c31178984aaa7"}
Oct 30 21:52:59.723801 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.723706    4412 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 21:52:59.724369 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.724356    4412 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-131-202.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Oct 30 21:52:59.724453 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.724380    4412 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-131-202.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 21:52:59.724453 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.724391    4412 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-131-202.us-west-1.compute.internal" event="NodeHasSufficientPID"
Oct 30 21:52:59.724544 ip-10-0-131-202 kubenswrapper[4412]: E1030 21:52:59.724518    4412 kubelet.go:3364] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-131-202.us-west-1.compute.internal\" not found" node="ip-10-0-131-202.us-west-1.compute.internal"
Oct 30 21:52:59.779542 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.779517    4412 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 21:52:59.780344 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.780330    4412 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-131-202.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Oct 30 21:52:59.780420 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.780358    4412 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-131-202.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 21:52:59.780420 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.780371    4412 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-131-202.us-west-1.compute.internal" event="NodeHasSufficientPID"
Oct 30 21:52:59.780420 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.780386    4412 kubelet_node_status.go:77] "Attempting to register node" node="ip-10-0-131-202.us-west-1.compute.internal"
Oct 30 21:52:59.781953 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.781932    4412 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 21:52:59.781953 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.781946    4412 reflector.go:446] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 21:52:59.782055 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.782001    4412 reconstruct.go:205] "DevicePaths of reconstructed volumes updated"
Oct 30 21:52:59.829886 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.829863    4412 reflector.go:446] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 21:52:59.832660 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.832646    4412 reflector.go:446] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 21:52:59.882773 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.882745    4412 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/04337a6f3666f0af07c3d2b702c6e379-config\") pod \"kube-apiserver-proxy-ip-10-0-131-202.us-west-1.compute.internal\" (UID: \"04337a6f3666f0af07c3d2b702c6e379\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-131-202.us-west-1.compute.internal"
Oct 30 21:52:59.882863 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.882774    4412 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/ec1e59c1a0a508a7c75743ab25eff8ba-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-131-202.us-west-1.compute.internal\" (UID: \"ec1e59c1a0a508a7c75743ab25eff8ba\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-202.us-west-1.compute.internal"
Oct 30 21:52:59.882863 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.882797    4412 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ec1e59c1a0a508a7c75743ab25eff8ba-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-131-202.us-west-1.compute.internal\" (UID: \"ec1e59c1a0a508a7c75743ab25eff8ba\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-202.us-west-1.compute.internal"
Oct 30 21:52:59.886923 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.886909    4412 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-202.us-west-1.compute.internal"
Oct 30 21:52:59.933222 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.933202    4412 kubelet_node_status.go:126] "Node was previously registered" node="ip-10-0-131-202.us-west-1.compute.internal"
Oct 30 21:52:59.933279 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.933266    4412 kubelet_node_status.go:80] "Successfully registered node" node="ip-10-0-131-202.us-west-1.compute.internal"
Oct 30 21:52:59.933814 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.933791    4412 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-131-202.us-west-1.compute.internal" event="NodeNotReady"
Oct 30 21:52:59.933905 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.933823    4412 setters.go:546] "Node became not ready" node="ip-10-0-131-202.us-west-1.compute.internal" condition={"type":"Ready","status":"False","lastHeartbeatTime":"2026-10-30T21:52:59Z","lastTransitionTime":"2026-10-30T21:52:59Z","reason":"KubeletNotReady","message":"CSINode is not yet initialized"}
Oct 30 21:52:59.947870 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.947851    4412 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 30 21:52:59.947949 ip-10-0-131-202 kubenswrapper[4412]: E1030 21:52:59.947887    4412 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-apiserver-proxy-ip-10-0-131-202.us-west-1.compute.internal\" already exists" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-202.us-west-1.compute.internal"
Oct 30 21:52:59.947949 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.947898    4412 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-202.us-west-1.compute.internal"
Oct 30 21:52:59.950034 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.950018    4412 nodeinfomanager.go:471] existing CSINode "ip-10-0-131-202.us-west-1.compute.internal" is owned by different node (oldNodeID="e0d4858f-03c1-45a1-855a-aa7efddca906", newNodeID=""), cleaning up...
Oct 30 21:52:59.983223 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.983172    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/04337a6f3666f0af07c3d2b702c6e379-config\") pod \"kube-apiserver-proxy-ip-10-0-131-202.us-west-1.compute.internal\" (UID: \"04337a6f3666f0af07c3d2b702c6e379\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-131-202.us-west-1.compute.internal"
Oct 30 21:52:59.983223 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.983206    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/ec1e59c1a0a508a7c75743ab25eff8ba-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-131-202.us-west-1.compute.internal\" (UID: \"ec1e59c1a0a508a7c75743ab25eff8ba\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-202.us-west-1.compute.internal"
Oct 30 21:52:59.983337 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.983230    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ec1e59c1a0a508a7c75743ab25eff8ba-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-131-202.us-west-1.compute.internal\" (UID: \"ec1e59c1a0a508a7c75743ab25eff8ba\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-202.us-west-1.compute.internal"
Oct 30 21:52:59.983337 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.983233    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/04337a6f3666f0af07c3d2b702c6e379-config\") pod \"kube-apiserver-proxy-ip-10-0-131-202.us-west-1.compute.internal\" (UID: \"04337a6f3666f0af07c3d2b702c6e379\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-131-202.us-west-1.compute.internal"
Oct 30 21:52:59.983337 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.983280    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/ec1e59c1a0a508a7c75743ab25eff8ba-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-131-202.us-west-1.compute.internal\" (UID: \"ec1e59c1a0a508a7c75743ab25eff8ba\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-202.us-west-1.compute.internal"
Oct 30 21:52:59.983337 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:52:59.983277    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ec1e59c1a0a508a7c75743ab25eff8ba-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-131-202.us-west-1.compute.internal\" (UID: \"ec1e59c1a0a508a7c75743ab25eff8ba\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-202.us-west-1.compute.internal"
Oct 30 21:53:00.049632 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.049604    4412 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 30 21:53:00.049755 ip-10-0-131-202 kubenswrapper[4412]: E1030 21:53:00.049654    4412 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-rbac-proxy-crio-ip-10-0-131-202.us-west-1.compute.internal\" already exists" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-202.us-west-1.compute.internal"
Oct 30 21:53:00.202983 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.202958    4412 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-131-202.us-west-1.compute.internal" event="NodeReady"
Oct 30 21:53:00.203221 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.203029    4412 kubelet_node_status.go:477] "Fast updating node status as it just became ready"
Oct 30 21:53:00.570172 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.570148    4412 apiserver.go:52] "Watching apiserver"
Oct 30 21:53:00.656597 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.656574    4412 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-202.us-west-1.compute.internal"
Oct 30 21:53:00.656780 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.656689    4412 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-202.us-west-1.compute.internal"
Oct 30 21:53:00.674935 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.674907    4412 reflector.go:446] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Oct 30 21:53:00.675346 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.675327    4412 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-target-slfvl","openshift-dns/node-resolver-7v7fb","openshift-network-operator/iptables-alerter-2hzq7","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nd9qk","openshift-multus/multus-hf5t4","kube-system/kube-apiserver-proxy-ip-10-0-131-202.us-west-1.compute.internal","openshift-insights/insights-runtime-extractor-bkhmz","openshift-kube-storage-version-migrator/migrator-66df5d496b-zbfpb","openshift-ovn-kubernetes/ovnkube-node-lhj9s","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-202.us-west-1.compute.internal","openshift-multus/network-metrics-daemon-wpdr9","kube-system/global-pull-secret-syncer-4vtdc","kube-system/konnectivity-agent-x7l5l","openshift-image-registry/node-ca-f9t6t","openshift-ingress-canary/ingress-canary-h2pjp","openshift-multus/multus-additional-cni-plugins-vpbvr","openshift-cluster-node-tuning-operator/tuned-9zvbw","openshift-dns/dns-default-8qxwh"]
Oct 30 21:53:00.676271 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.676255    4412 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wpdr9"
Oct 30 21:53:00.682246 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.682221    4412 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-8qxwh"
Oct 30 21:53:00.682715 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.682680    4412 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-bkhmz"
Oct 30 21:53:00.686467 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.686447    4412 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Oct 30 21:53:00.688640 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.688623    4412 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/bf0e36d0-87bd-4e1e-be40-8a3a0f856987-host-cni-bin\") pod \"ovnkube-node-lhj9s\" (UID: \"bf0e36d0-87bd-4e1e-be40-8a3a0f856987\") " pod="openshift-ovn-kubernetes/ovnkube-node-lhj9s"
Oct 30 21:53:00.688699 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.688650    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dffd7a70-b7f3-4e1b-b3ab-74c5a574cab2-tmp-dir\") pod \"node-resolver-7v7fb\" (UID: \"dffd7a70-b7f3-4e1b-b3ab-74c5a574cab2\") " pod="openshift-dns/node-resolver-7v7fb"
Oct 30 21:53:00.688699 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.688668    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hrlj5\" (UniqueName: \"kubernetes.io/projected/dffd7a70-b7f3-4e1b-b3ab-74c5a574cab2-kube-api-access-hrlj5\") pod \"node-resolver-7v7fb\" (UID: \"dffd7a70-b7f3-4e1b-b3ab-74c5a574cab2\") " pod="openshift-dns/node-resolver-7v7fb"
Oct 30 21:53:00.688699 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.688682    4412 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/982efb96-3613-4ef2-ac51-ed3f40745a32-etc-kubernetes\") pod \"tuned-9zvbw\" (UID: \"982efb96-3613-4ef2-ac51-ed3f40745a32\") " pod="openshift-cluster-node-tuning-operator/tuned-9zvbw"
Oct 30 21:53:00.688792 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.688777    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dffd7a70-b7f3-4e1b-b3ab-74c5a574cab2-tmp-dir\") pod \"node-resolver-7v7fb\" (UID: \"dffd7a70-b7f3-4e1b-b3ab-74c5a574cab2\") " pod="openshift-dns/node-resolver-7v7fb"
Oct 30 21:53:00.688949 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.688935    4412 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/d17b7754-d1d4-4952-97c7-7a04c714b48c-kubelet-config\") pod \"global-pull-secret-syncer-4vtdc\" (UID: \"d17b7754-d1d4-4952-97c7-7a04c714b48c\") " pod="kube-system/global-pull-secret-syncer-4vtdc"
Oct 30 21:53:00.688984 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.688962    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d17b7754-d1d4-4952-97c7-7a04c714b48c-original-pull-secret\") pod \"global-pull-secret-syncer-4vtdc\" (UID: \"d17b7754-d1d4-4952-97c7-7a04c714b48c\") " pod="kube-system/global-pull-secret-syncer-4vtdc"
Oct 30 21:53:00.688984 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.688978    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/bf0e36d0-87bd-4e1e-be40-8a3a0f856987-env-overrides\") pod \"ovnkube-node-lhj9s\" (UID: \"bf0e36d0-87bd-4e1e-be40-8a3a0f856987\") " pod="openshift-ovn-kubernetes/ovnkube-node-lhj9s"
Oct 30 21:53:00.689049 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.688994    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/bf0e36d0-87bd-4e1e-be40-8a3a0f856987-ovnkube-script-lib\") pod \"ovnkube-node-lhj9s\" (UID: \"bf0e36d0-87bd-4e1e-be40-8a3a0f856987\") " pod="openshift-ovn-kubernetes/ovnkube-node-lhj9s"
Oct 30 21:53:00.689049 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.689015    4412 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/db073c3b-7cee-4e22-8c11-209bbeb21a96-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-bkhmz\" (UID: \"db073c3b-7cee-4e22-8c11-209bbeb21a96\") " pod="openshift-insights/insights-runtime-extractor-bkhmz"
Oct 30 21:53:00.689049 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.689033    4412 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/164bafd8-ff17-46a2-afc0-dd01fa1aaeb6-host\") pod \"node-ca-f9t6t\" (UID: \"164bafd8-ff17-46a2-afc0-dd01fa1aaeb6\") " pod="openshift-image-registry/node-ca-f9t6t"
Oct 30 21:53:00.689198 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.689055    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6j99v\" (UniqueName: \"kubernetes.io/projected/7be00c31-281f-4d1c-8e12-1bf74bbe7cbd-kube-api-access-6j99v\") pod \"network-check-target-slfvl\" (UID: \"7be00c31-281f-4d1c-8e12-1bf74bbe7cbd\") " pod="openshift-network-diagnostics/network-check-target-slfvl"
Oct 30 21:53:00.689198 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.689100    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ctg2b\" (UniqueName: \"kubernetes.io/projected/bf0e36d0-87bd-4e1e-be40-8a3a0f856987-kube-api-access-ctg2b\") pod \"ovnkube-node-lhj9s\" (UID: \"bf0e36d0-87bd-4e1e-be40-8a3a0f856987\") " pod="openshift-ovn-kubernetes/ovnkube-node-lhj9s"
Oct 30 21:53:00.689198 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.689167    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/a4ff4cce-671b-4136-8b63-b22148c0db4e-iptables-alerter-script\") pod \"iptables-alerter-2hzq7\" (UID: \"a4ff4cce-671b-4136-8b63-b22148c0db4e\") " pod="openshift-network-operator/iptables-alerter-2hzq7"
Oct 30 21:53:00.689305 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.689196    4412 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a4ff4cce-671b-4136-8b63-b22148c0db4e-host-slash\") pod \"iptables-alerter-2hzq7\" (UID: \"a4ff4cce-671b-4136-8b63-b22148c0db4e\") " pod="openshift-network-operator/iptables-alerter-2hzq7"
Oct 30 21:53:00.689305 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.689223    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/164bafd8-ff17-46a2-afc0-dd01fa1aaeb6-serviceca\") pod \"node-ca-f9t6t\" (UID: \"164bafd8-ff17-46a2-afc0-dd01fa1aaeb6\") " pod="openshift-image-registry/node-ca-f9t6t"
Oct 30 21:53:00.689305 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.689255    4412 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c590d5fe-57de-4905-90fe-8527b215ce94-cnibin\") pod \"multus-additional-cni-plugins-vpbvr\" (UID: \"c590d5fe-57de-4905-90fe-8527b215ce94\") " pod="openshift-multus/multus-additional-cni-plugins-vpbvr"
Oct 30 21:53:00.689305 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.689276    4412 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b1a02001-9121-47bd-ae96-b39196ad2357-cnibin\") pod \"multus-hf5t4\" (UID: \"b1a02001-9121-47bd-ae96-b39196ad2357\") " pod="openshift-multus/multus-hf5t4"
Oct 30 21:53:00.689305 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.689292    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b1a02001-9121-47bd-ae96-b39196ad2357-cni-binary-copy\") pod \"multus-hf5t4\" (UID: \"b1a02001-9121-47bd-ae96-b39196ad2357\") " pod="openshift-multus/multus-hf5t4"
Oct 30 21:53:00.689480 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.689306    4412 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/b1a02001-9121-47bd-ae96-b39196ad2357-hostroot\") pod \"multus-hf5t4\" (UID: \"b1a02001-9121-47bd-ae96-b39196ad2357\") " pod="openshift-multus/multus-hf5t4"
Oct 30 21:53:00.689480 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.689340    4412 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b1a02001-9121-47bd-ae96-b39196ad2357-multus-conf-dir\") pod \"multus-hf5t4\" (UID: \"b1a02001-9121-47bd-ae96-b39196ad2357\") " pod="openshift-multus/multus-hf5t4"
Oct 30 21:53:00.689480 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.689363    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4a03a836-506f-404e-9c55-c26d779cd4ee-tmp-dir\") pod \"dns-default-8qxwh\" (UID: \"4a03a836-506f-404e-9c55-c26d779cd4ee\") " pod="openshift-dns/dns-default-8qxwh"
Oct 30 21:53:00.689480 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.689379    4412 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/982efb96-3613-4ef2-ac51-ed3f40745a32-etc-sysconfig\") pod \"tuned-9zvbw\" (UID: \"982efb96-3613-4ef2-ac51-ed3f40745a32\") " pod="openshift-cluster-node-tuning-operator/tuned-9zvbw"
Oct 30 21:53:00.689480 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.689395    4412 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bf0e36d0-87bd-4e1e-be40-8a3a0f856987-var-lib-openvswitch\") pod \"ovnkube-node-lhj9s\" (UID: \"bf0e36d0-87bd-4e1e-be40-8a3a0f856987\") " pod="openshift-ovn-kubernetes/ovnkube-node-lhj9s"
Oct 30 21:53:00.689480 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.689409    4412 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bf0e36d0-87bd-4e1e-be40-8a3a0f856987-etc-openvswitch\") pod \"ovnkube-node-lhj9s\" (UID: \"bf0e36d0-87bd-4e1e-be40-8a3a0f856987\") " pod="openshift-ovn-kubernetes/ovnkube-node-lhj9s"
Oct 30 21:53:00.689480 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.689423    4412 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b1a02001-9121-47bd-ae96-b39196ad2357-host-var-lib-cni-bin\") pod \"multus-hf5t4\" (UID: \"b1a02001-9121-47bd-ae96-b39196ad2357\") " pod="openshift-multus/multus-hf5t4"
Oct 30 21:53:00.689480 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.689465    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vsg28\" (UniqueName: \"kubernetes.io/projected/b1a02001-9121-47bd-ae96-b39196ad2357-kube-api-access-vsg28\") pod \"multus-hf5t4\" (UID: \"b1a02001-9121-47bd-ae96-b39196ad2357\") " pod="openshift-multus/multus-hf5t4"
Oct 30 21:53:00.689766 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.689538    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4a03a836-506f-404e-9c55-c26d779cd4ee-tmp-dir\") pod \"dns-default-8qxwh\" (UID: \"4a03a836-506f-404e-9c55-c26d779cd4ee\") " pod="openshift-dns/dns-default-8qxwh"
Oct 30 21:53:00.689766 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.689533    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fsf4l\" (UniqueName: \"kubernetes.io/projected/7e6b0df0-07eb-4301-adaa-1a6514639d9c-kube-api-access-fsf4l\") pod \"migrator-66df5d496b-zbfpb\" (UID: \"7e6b0df0-07eb-4301-adaa-1a6514639d9c\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-zbfpb"
Oct 30 21:53:00.689766 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.689585    4412 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/982efb96-3613-4ef2-ac51-ed3f40745a32-sys\") pod \"tuned-9zvbw\" (UID: \"982efb96-3613-4ef2-ac51-ed3f40745a32\") " pod="openshift-cluster-node-tuning-operator/tuned-9zvbw"
Oct 30 21:53:00.689766 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.689625    4412 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/982efb96-3613-4ef2-ac51-ed3f40745a32-var-lib-kubelet\") pod \"tuned-9zvbw\" (UID: \"982efb96-3613-4ef2-ac51-ed3f40745a32\") " pod="openshift-cluster-node-tuning-operator/tuned-9zvbw"
Oct 30 21:53:00.689766 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.689659    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vd7d5\" (UniqueName: \"kubernetes.io/projected/db073c3b-7cee-4e22-8c11-209bbeb21a96-kube-api-access-vd7d5\") pod \"insights-runtime-extractor-bkhmz\" (UID: \"db073c3b-7cee-4e22-8c11-209bbeb21a96\") " pod="openshift-insights/insights-runtime-extractor-bkhmz"
Oct 30 21:53:00.689766 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.689683    4412 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/982efb96-3613-4ef2-ac51-ed3f40745a32-etc-sysctl-conf\") pod \"tuned-9zvbw\" (UID: \"982efb96-3613-4ef2-ac51-ed3f40745a32\") " pod="openshift-cluster-node-tuning-operator/tuned-9zvbw"
Oct 30 21:53:00.689766 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.689742    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c622t\" (UniqueName: \"kubernetes.io/projected/2345ff5b-db97-4855-8adb-59b9a780615e-kube-api-access-c622t\") pod \"ingress-canary-h2pjp\" (UID: \"2345ff5b-db97-4855-8adb-59b9a780615e\") " pod="openshift-ingress-canary/ingress-canary-h2pjp"
Oct 30 21:53:00.689766 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.689765    4412 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b1a02001-9121-47bd-ae96-b39196ad2357-multus-cni-dir\") pod \"multus-hf5t4\" (UID: \"b1a02001-9121-47bd-ae96-b39196ad2357\") " pod="openshift-multus/multus-hf5t4"
Oct 30 21:53:00.690168 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.689780    4412 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/b1a02001-9121-47bd-ae96-b39196ad2357-host-var-lib-cni-multus\") pod \"multus-hf5t4\" (UID: \"b1a02001-9121-47bd-ae96-b39196ad2357\") " pod="openshift-multus/multus-hf5t4"
Oct 30 21:53:00.690168 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.689797    4412 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bf0e36d0-87bd-4e1e-be40-8a3a0f856987-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-lhj9s\" (UID: \"bf0e36d0-87bd-4e1e-be40-8a3a0f856987\") " pod="openshift-ovn-kubernetes/ovnkube-node-lhj9s"
Oct 30 21:53:00.690168 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.689814    4412 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/386f9569-a5f5-4a9e-beee-3005994186cd-device-dir\") pod \"aws-ebs-csi-driver-node-nd9qk\" (UID: \"386f9569-a5f5-4a9e-beee-3005994186cd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nd9qk"
Oct 30 21:53:00.690168 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.689854    4412 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/386f9569-a5f5-4a9e-beee-3005994186cd-sys-fs\") pod \"aws-ebs-csi-driver-node-nd9qk\" (UID: \"386f9569-a5f5-4a9e-beee-3005994186cd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nd9qk"
Oct 30 21:53:00.690168 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.689884    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/c590d5fe-57de-4905-90fe-8527b215ce94-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-vpbvr\" (UID: \"c590d5fe-57de-4905-90fe-8527b215ce94\") " pod="openshift-multus/multus-additional-cni-plugins-vpbvr"
Oct 30 21:53:00.690168 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.689910    4412 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/bf0e36d0-87bd-4e1e-be40-8a3a0f856987-host-run-netns\") pod \"ovnkube-node-lhj9s\" (UID: \"bf0e36d0-87bd-4e1e-be40-8a3a0f856987\") " pod="openshift-ovn-kubernetes/ovnkube-node-lhj9s"
Oct 30 21:53:00.690168 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.689944    4412 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/dffd7a70-b7f3-4e1b-b3ab-74c5a574cab2-hosts-file\") pod \"node-resolver-7v7fb\" (UID: \"dffd7a70-b7f3-4e1b-b3ab-74c5a574cab2\") " pod="openshift-dns/node-resolver-7v7fb"
Oct 30 21:53:00.690168 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.689970    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/25590e1f-99e3-4469-8a69-39cd30317c28-agent-certs\") pod \"konnectivity-agent-x7l5l\" (UID: \"25590e1f-99e3-4469-8a69-39cd30317c28\") " pod="kube-system/konnectivity-agent-x7l5l"
Oct 30 21:53:00.690168 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.689998    4412 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8806f1e1-0057-4738-8fbc-7e503e1f91ca-metrics-certs\") pod \"network-metrics-daemon-wpdr9\" (UID: \"8806f1e1-0057-4738-8fbc-7e503e1f91ca\") " pod="openshift-multus/network-metrics-daemon-wpdr9"
Oct 30 21:53:00.690168 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.690025    4412 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c590d5fe-57de-4905-90fe-8527b215ce94-system-cni-dir\") pod \"multus-additional-cni-plugins-vpbvr\" (UID: \"c590d5fe-57de-4905-90fe-8527b215ce94\") " pod="openshift-multus/multus-additional-cni-plugins-vpbvr"
Oct 30 21:53:00.690168 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.690049    4412 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bf0e36d0-87bd-4e1e-be40-8a3a0f856987-host-run-ovn-kubernetes\") pod \"ovnkube-node-lhj9s\" (UID: \"bf0e36d0-87bd-4e1e-be40-8a3a0f856987\") " pod="openshift-ovn-kubernetes/ovnkube-node-lhj9s"
Oct 30 21:53:00.690168 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.690072    4412 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/bf0e36d0-87bd-4e1e-be40-8a3a0f856987-host-cni-netd\") pod \"ovnkube-node-lhj9s\" (UID: \"bf0e36d0-87bd-4e1e-be40-8a3a0f856987\") " pod="openshift-ovn-kubernetes/ovnkube-node-lhj9s"
Oct 30 21:53:00.690168 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.690115    4412 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/b1a02001-9121-47bd-ae96-b39196ad2357-multus-socket-dir-parent\") pod \"multus-hf5t4\" (UID: \"b1a02001-9121-47bd-ae96-b39196ad2357\") " pod="openshift-multus/multus-hf5t4"
Oct 30 21:53:00.690168 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.690161    4412 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/b1a02001-9121-47bd-ae96-b39196ad2357-host-run-k8s-cni-cncf-io\") pod \"multus-hf5t4\" (UID: \"b1a02001-9121-47bd-ae96-b39196ad2357\") " pod="openshift-multus/multus-hf5t4"
Oct 30 21:53:00.690571 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.690176    4412 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/db073c3b-7cee-4e22-8c11-209bbeb21a96-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-bkhmz\" (UID: \"db073c3b-7cee-4e22-8c11-209bbeb21a96\") " pod="openshift-insights/insights-runtime-extractor-bkhmz"
Oct 30 21:53:00.690571 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.690197    4412 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/386f9569-a5f5-4a9e-beee-3005994186cd-registration-dir\") pod \"aws-ebs-csi-driver-node-nd9qk\" (UID: \"386f9569-a5f5-4a9e-beee-3005994186cd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nd9qk"
Oct 30 21:53:00.690571 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.690227    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/bf0e36d0-87bd-4e1e-be40-8a3a0f856987-ovnkube-config\") pod \"ovnkube-node-lhj9s\" (UID: \"bf0e36d0-87bd-4e1e-be40-8a3a0f856987\") " pod="openshift-ovn-kubernetes/ovnkube-node-lhj9s"
Oct 30 21:53:00.690571 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.690257    4412 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b1a02001-9121-47bd-ae96-b39196ad2357-host-var-lib-kubelet\") pod \"multus-hf5t4\" (UID: \"b1a02001-9121-47bd-ae96-b39196ad2357\") " pod="openshift-multus/multus-hf5t4"
Oct 30 21:53:00.690571 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.690294    4412 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/b1a02001-9121-47bd-ae96-b39196ad2357-host-run-multus-certs\") pod \"multus-hf5t4\" (UID: \"b1a02001-9121-47bd-ae96-b39196ad2357\") " pod="openshift-multus/multus-hf5t4"
Oct 30 21:53:00.690571 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.690322    4412 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b1a02001-9121-47bd-ae96-b39196ad2357-etc-kubernetes\") pod \"multus-hf5t4\" (UID: \"b1a02001-9121-47bd-ae96-b39196ad2357\") " pod="openshift-multus/multus-hf5t4"
Oct 30 21:53:00.690571 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.690339    4412 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/982efb96-3613-4ef2-ac51-ed3f40745a32-etc-modprobe-d\") pod \"tuned-9zvbw\" (UID: \"982efb96-3613-4ef2-ac51-ed3f40745a32\") " pod="openshift-cluster-node-tuning-operator/tuned-9zvbw"
Oct 30 21:53:00.690571 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.690359    4412 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/982efb96-3613-4ef2-ac51-ed3f40745a32-etc-sysctl-d\") pod \"tuned-9zvbw\" (UID: \"982efb96-3613-4ef2-ac51-ed3f40745a32\") " pod="openshift-cluster-node-tuning-operator/tuned-9zvbw"
Oct 30 21:53:00.690571 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.690373    4412 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/982efb96-3613-4ef2-ac51-ed3f40745a32-etc-systemd\") pod \"tuned-9zvbw\" (UID: \"982efb96-3613-4ef2-ac51-ed3f40745a32\") " pod="openshift-cluster-node-tuning-operator/tuned-9zvbw"
Oct 30 21:53:00.690571 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.690397    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/db073c3b-7cee-4e22-8c11-209bbeb21a96-data-volume\") pod \"insights-runtime-extractor-bkhmz\" (UID: \"db073c3b-7cee-4e22-8c11-209bbeb21a96\") " pod="openshift-insights/insights-runtime-extractor-bkhmz"
Oct 30 21:53:00.690571 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.690429    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/c590d5fe-57de-4905-90fe-8527b215ce94-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-vpbvr\" (UID: \"c590d5fe-57de-4905-90fe-8527b215ce94\") " pod="openshift-multus/multus-additional-cni-plugins-vpbvr"
Oct 30 21:53:00.690571 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.690455    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/db073c3b-7cee-4e22-8c11-209bbeb21a96-data-volume\") pod \"insights-runtime-extractor-bkhmz\" (UID: \"db073c3b-7cee-4e22-8c11-209bbeb21a96\") " pod="openshift-insights/insights-runtime-extractor-bkhmz"
Oct 30 21:53:00.690571 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.690469    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/982efb96-3613-4ef2-ac51-ed3f40745a32-etc-tuned\") pod \"tuned-9zvbw\" (UID: \"982efb96-3613-4ef2-ac51-ed3f40745a32\") " pod="openshift-cluster-node-tuning-operator/tuned-9zvbw"
Oct 30 21:53:00.690571 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.690504    4412 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/386f9569-a5f5-4a9e-beee-3005994186cd-etc-selinux\") pod \"aws-ebs-csi-driver-node-nd9qk\" (UID: \"386f9569-a5f5-4a9e-beee-3005994186cd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nd9qk"
Oct 30 21:53:00.690571 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.690524    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/982efb96-3613-4ef2-ac51-ed3f40745a32-etc-tuned\") pod \"tuned-9zvbw\" (UID: \"982efb96-3613-4ef2-ac51-ed3f40745a32\") " pod="openshift-cluster-node-tuning-operator/tuned-9zvbw"
Oct 30 21:53:00.690571 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.690532    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2345ff5b-db97-4855-8adb-59b9a780615e-cert\") pod \"ingress-canary-h2pjp\" (UID: \"2345ff5b-db97-4855-8adb-59b9a780615e\") " pod="openshift-ingress-canary/ingress-canary-h2pjp"
Oct 30 21:53:00.690571 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.690548    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fwsxm\" (UniqueName: \"kubernetes.io/projected/8806f1e1-0057-4738-8fbc-7e503e1f91ca-kube-api-access-fwsxm\") pod \"network-metrics-daemon-wpdr9\" (UID: \"8806f1e1-0057-4738-8fbc-7e503e1f91ca\") " pod="openshift-multus/network-metrics-daemon-wpdr9"
Oct 30 21:53:00.691241 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.690599    4412 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/bf0e36d0-87bd-4e1e-be40-8a3a0f856987-systemd-units\") pod \"ovnkube-node-lhj9s\" (UID: \"bf0e36d0-87bd-4e1e-be40-8a3a0f856987\") " pod="openshift-ovn-kubernetes/ovnkube-node-lhj9s"
Oct 30 21:53:00.691241 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.690615    4412 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/bf0e36d0-87bd-4e1e-be40-8a3a0f856987-run-systemd\") pod \"ovnkube-node-lhj9s\" (UID: \"bf0e36d0-87bd-4e1e-be40-8a3a0f856987\") " pod="openshift-ovn-kubernetes/ovnkube-node-lhj9s"
Oct 30 21:53:00.691241 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.690743    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/bf0e36d0-87bd-4e1e-be40-8a3a0f856987-ovn-node-metrics-cert\") pod \"ovnkube-node-lhj9s\" (UID: \"bf0e36d0-87bd-4e1e-be40-8a3a0f856987\") " pod="openshift-ovn-kubernetes/ovnkube-node-lhj9s"
Oct 30 21:53:00.691241 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.690761    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/b1a02001-9121-47bd-ae96-b39196ad2357-multus-daemon-config\") pod \"multus-hf5t4\" (UID: \"b1a02001-9121-47bd-ae96-b39196ad2357\") " pod="openshift-multus/multus-hf5t4"
Oct 30 21:53:00.691241 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.690775    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2pwzh\" (UniqueName: \"kubernetes.io/projected/982efb96-3613-4ef2-ac51-ed3f40745a32-kube-api-access-2pwzh\") pod \"tuned-9zvbw\" (UID: \"982efb96-3613-4ef2-ac51-ed3f40745a32\") " pod="openshift-cluster-node-tuning-operator/tuned-9zvbw"
Oct 30 21:53:00.691241 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.690866    4412 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/db073c3b-7cee-4e22-8c11-209bbeb21a96-crio-socket\") pod \"insights-runtime-extractor-bkhmz\" (UID: \"db073c3b-7cee-4e22-8c11-209bbeb21a96\") " pod="openshift-insights/insights-runtime-extractor-bkhmz"
Oct 30 21:53:00.691241 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.690905    4412 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/386f9569-a5f5-4a9e-beee-3005994186cd-kubelet-dir\") pod \"aws-ebs-csi-driver-node-nd9qk\" (UID: \"386f9569-a5f5-4a9e-beee-3005994186cd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nd9qk"
Oct 30 21:53:00.691241 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.690930    4412 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b1a02001-9121-47bd-ae96-b39196ad2357-system-cni-dir\") pod \"multus-hf5t4\" (UID: \"b1a02001-9121-47bd-ae96-b39196ad2357\") " pod="openshift-multus/multus-hf5t4"
Oct 30 21:53:00.691241 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.690955    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/4a03a836-506f-404e-9c55-c26d779cd4ee-config-volume\") pod \"dns-default-8qxwh\" (UID: \"4a03a836-506f-404e-9c55-c26d779cd4ee\") " pod="openshift-dns/dns-default-8qxwh"
Oct 30 21:53:00.691241 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.690996    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/982efb96-3613-4ef2-ac51-ed3f40745a32-tmp\") pod \"tuned-9zvbw\" (UID: \"982efb96-3613-4ef2-ac51-ed3f40745a32\") " pod="openshift-cluster-node-tuning-operator/tuned-9zvbw"
Oct 30 21:53:00.691241 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.691022    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c590d5fe-57de-4905-90fe-8527b215ce94-cni-binary-copy\") pod \"multus-additional-cni-plugins-vpbvr\" (UID: \"c590d5fe-57de-4905-90fe-8527b215ce94\") " pod="openshift-multus/multus-additional-cni-plugins-vpbvr"
Oct 30 21:53:00.691241 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.691039    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/982efb96-3613-4ef2-ac51-ed3f40745a32-tmp\") pod \"tuned-9zvbw\" (UID: \"982efb96-3613-4ef2-ac51-ed3f40745a32\") " pod="openshift-cluster-node-tuning-operator/tuned-9zvbw"
Oct 30 21:53:00.691241 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.691040    4412 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/bf0e36d0-87bd-4e1e-be40-8a3a0f856987-log-socket\") pod \"ovnkube-node-lhj9s\" (UID: \"bf0e36d0-87bd-4e1e-be40-8a3a0f856987\") " pod="openshift-ovn-kubernetes/ovnkube-node-lhj9s"
Oct 30 21:53:00.691241 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.691060    4412 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/982efb96-3613-4ef2-ac51-ed3f40745a32-run\") pod \"tuned-9zvbw\" (UID: \"982efb96-3613-4ef2-ac51-ed3f40745a32\") " pod="openshift-cluster-node-tuning-operator/tuned-9zvbw"
Oct 30 21:53:00.691241 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.691084    4412 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/982efb96-3613-4ef2-ac51-ed3f40745a32-lib-modules\") pod \"tuned-9zvbw\" (UID: \"982efb96-3613-4ef2-ac51-ed3f40745a32\") " pod="openshift-cluster-node-tuning-operator/tuned-9zvbw"
Oct 30 21:53:00.691241 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.691100    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tg4f9\" (UniqueName: \"kubernetes.io/projected/386f9569-a5f5-4a9e-beee-3005994186cd-kube-api-access-tg4f9\") pod \"aws-ebs-csi-driver-node-nd9qk\" (UID: \"386f9569-a5f5-4a9e-beee-3005994186cd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nd9qk"
Oct 30 21:53:00.691241 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.691149    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6r8d9\" (UniqueName: \"kubernetes.io/projected/a4ff4cce-671b-4136-8b63-b22148c0db4e-kube-api-access-6r8d9\") pod \"iptables-alerter-2hzq7\" (UID: \"a4ff4cce-671b-4136-8b63-b22148c0db4e\") " pod="openshift-network-operator/iptables-alerter-2hzq7"
Oct 30 21:53:00.691241 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.691167    4412 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bf0e36d0-87bd-4e1e-be40-8a3a0f856987-run-openvswitch\") pod \"ovnkube-node-lhj9s\" (UID: \"bf0e36d0-87bd-4e1e-be40-8a3a0f856987\") " pod="openshift-ovn-kubernetes/ovnkube-node-lhj9s"
Oct 30 21:53:00.691241 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.691246    4412 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/bf0e36d0-87bd-4e1e-be40-8a3a0f856987-node-log\") pod \"ovnkube-node-lhj9s\" (UID: \"bf0e36d0-87bd-4e1e-be40-8a3a0f856987\") " pod="openshift-ovn-kubernetes/ovnkube-node-lhj9s"
Oct 30 21:53:00.691947 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.691272    4412 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/bf0e36d0-87bd-4e1e-be40-8a3a0f856987-host-kubelet\") pod \"ovnkube-node-lhj9s\" (UID: \"bf0e36d0-87bd-4e1e-be40-8a3a0f856987\") " pod="openshift-ovn-kubernetes/ovnkube-node-lhj9s"
Oct 30 21:53:00.691947 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.691291    4412 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c590d5fe-57de-4905-90fe-8527b215ce94-tuning-conf-dir\") pod \"multus-additional-cni-plugins-vpbvr\" (UID: \"c590d5fe-57de-4905-90fe-8527b215ce94\") " pod="openshift-multus/multus-additional-cni-plugins-vpbvr"
Oct 30 21:53:00.691947 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.691324    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fq868\" (UniqueName: \"kubernetes.io/projected/c590d5fe-57de-4905-90fe-8527b215ce94-kube-api-access-fq868\") pod \"multus-additional-cni-plugins-vpbvr\" (UID: \"c590d5fe-57de-4905-90fe-8527b215ce94\") " pod="openshift-multus/multus-additional-cni-plugins-vpbvr"
Oct 30 21:53:00.691947 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.691349    4412 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/bf0e36d0-87bd-4e1e-be40-8a3a0f856987-host-slash\") pod \"ovnkube-node-lhj9s\" (UID: \"bf0e36d0-87bd-4e1e-be40-8a3a0f856987\") " pod="openshift-ovn-kubernetes/ovnkube-node-lhj9s"
Oct 30 21:53:00.691947 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.691453    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dtsnb\" (UniqueName: \"kubernetes.io/projected/4a03a836-506f-404e-9c55-c26d779cd4ee-kube-api-access-dtsnb\") pod \"dns-default-8qxwh\" (UID: \"4a03a836-506f-404e-9c55-c26d779cd4ee\") " pod="openshift-dns/dns-default-8qxwh"
Oct 30 21:53:00.691947 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.691471    4412 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/bf0e36d0-87bd-4e1e-be40-8a3a0f856987-run-ovn\") pod \"ovnkube-node-lhj9s\" (UID: \"bf0e36d0-87bd-4e1e-be40-8a3a0f856987\") " pod="openshift-ovn-kubernetes/ovnkube-node-lhj9s"
Oct 30 21:53:00.691947 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.691487    4412 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b1a02001-9121-47bd-ae96-b39196ad2357-host-run-netns\") pod \"multus-hf5t4\" (UID: \"b1a02001-9121-47bd-ae96-b39196ad2357\") " pod="openshift-multus/multus-hf5t4"
Oct 30 21:53:00.691947 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.691502    4412 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4a03a836-506f-404e-9c55-c26d779cd4ee-metrics-tls\") pod \"dns-default-8qxwh\" (UID: \"4a03a836-506f-404e-9c55-c26d779cd4ee\") " pod="openshift-dns/dns-default-8qxwh"
Oct 30 21:53:00.691947 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.691516    4412 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/982efb96-3613-4ef2-ac51-ed3f40745a32-host\") pod \"tuned-9zvbw\" (UID: \"982efb96-3613-4ef2-ac51-ed3f40745a32\") " pod="openshift-cluster-node-tuning-operator/tuned-9zvbw"
Oct 30 21:53:00.691947 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.691531    4412 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/d17b7754-d1d4-4952-97c7-7a04c714b48c-dbus\") pod \"global-pull-secret-syncer-4vtdc\" (UID: \"d17b7754-d1d4-4952-97c7-7a04c714b48c\") " pod="kube-system/global-pull-secret-syncer-4vtdc"
Oct 30 21:53:00.691947 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.691615    4412 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/386f9569-a5f5-4a9e-beee-3005994186cd-socket-dir\") pod \"aws-ebs-csi-driver-node-nd9qk\" (UID: \"386f9569-a5f5-4a9e-beee-3005994186cd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nd9qk"
Oct 30 21:53:00.691947 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.691642    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/25590e1f-99e3-4469-8a69-39cd30317c28-konnectivity-ca\") pod \"konnectivity-agent-x7l5l\" (UID: \"25590e1f-99e3-4469-8a69-39cd30317c28\") " pod="kube-system/konnectivity-agent-x7l5l"
Oct 30 21:53:00.691947 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.691661    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-78t6j\" (UniqueName: \"kubernetes.io/projected/164bafd8-ff17-46a2-afc0-dd01fa1aaeb6-kube-api-access-78t6j\") pod \"node-ca-f9t6t\" (UID: \"164bafd8-ff17-46a2-afc0-dd01fa1aaeb6\") " pod="openshift-image-registry/node-ca-f9t6t"
Oct 30 21:53:00.718347 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.718327    4412 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 30 21:53:00.718409 ip-10-0-131-202 kubenswrapper[4412]: E1030 21:53:00.718366    4412 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-apiserver-proxy-ip-10-0-131-202.us-west-1.compute.internal\" already exists" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-202.us-west-1.compute.internal"
Oct 30 21:53:00.718869 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.718857    4412 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 30 21:53:00.718903 ip-10-0-131-202 kubenswrapper[4412]: E1030 21:53:00.718882    4412 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-rbac-proxy-crio-ip-10-0-131-202.us-west-1.compute.internal\" already exists" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-202.us-west-1.compute.internal"
Oct 30 21:53:00.731882 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.731859    4412 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Oct 30 21:53:00.732016 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.732001    4412 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Oct 30 21:53:00.732819 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.732804    4412 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-q4htg\""
Oct 30 21:53:00.733419 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.733406    4412 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Oct 30 21:53:00.791926 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.791898    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b1a02001-9121-47bd-ae96-b39196ad2357-system-cni-dir\") pod \"multus-hf5t4\" (UID: \"b1a02001-9121-47bd-ae96-b39196ad2357\") " pod="openshift-multus/multus-hf5t4"
Oct 30 21:53:00.792028 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.791931    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/bf0e36d0-87bd-4e1e-be40-8a3a0f856987-log-socket\") pod \"ovnkube-node-lhj9s\" (UID: \"bf0e36d0-87bd-4e1e-be40-8a3a0f856987\") " pod="openshift-ovn-kubernetes/ovnkube-node-lhj9s"
Oct 30 21:53:00.792028 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.791948    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/982efb96-3613-4ef2-ac51-ed3f40745a32-run\") pod \"tuned-9zvbw\" (UID: \"982efb96-3613-4ef2-ac51-ed3f40745a32\") " pod="openshift-cluster-node-tuning-operator/tuned-9zvbw"
Oct 30 21:53:00.792028 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.791963    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/982efb96-3613-4ef2-ac51-ed3f40745a32-lib-modules\") pod \"tuned-9zvbw\" (UID: \"982efb96-3613-4ef2-ac51-ed3f40745a32\") " pod="openshift-cluster-node-tuning-operator/tuned-9zvbw"
Oct 30 21:53:00.792028 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.792018    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/bf0e36d0-87bd-4e1e-be40-8a3a0f856987-log-socket\") pod \"ovnkube-node-lhj9s\" (UID: \"bf0e36d0-87bd-4e1e-be40-8a3a0f856987\") " pod="openshift-ovn-kubernetes/ovnkube-node-lhj9s"
Oct 30 21:53:00.792269 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.792030    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/982efb96-3613-4ef2-ac51-ed3f40745a32-run\") pod \"tuned-9zvbw\" (UID: \"982efb96-3613-4ef2-ac51-ed3f40745a32\") " pod="openshift-cluster-node-tuning-operator/tuned-9zvbw"
Oct 30 21:53:00.792269 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.792028    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b1a02001-9121-47bd-ae96-b39196ad2357-system-cni-dir\") pod \"multus-hf5t4\" (UID: \"b1a02001-9121-47bd-ae96-b39196ad2357\") " pod="openshift-multus/multus-hf5t4"
Oct 30 21:53:00.792269 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.792052    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bf0e36d0-87bd-4e1e-be40-8a3a0f856987-run-openvswitch\") pod \"ovnkube-node-lhj9s\" (UID: \"bf0e36d0-87bd-4e1e-be40-8a3a0f856987\") " pod="openshift-ovn-kubernetes/ovnkube-node-lhj9s"
Oct 30 21:53:00.792269 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.792074    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/982efb96-3613-4ef2-ac51-ed3f40745a32-lib-modules\") pod \"tuned-9zvbw\" (UID: \"982efb96-3613-4ef2-ac51-ed3f40745a32\") " pod="openshift-cluster-node-tuning-operator/tuned-9zvbw"
Oct 30 21:53:00.792269 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.792085    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/bf0e36d0-87bd-4e1e-be40-8a3a0f856987-node-log\") pod \"ovnkube-node-lhj9s\" (UID: \"bf0e36d0-87bd-4e1e-be40-8a3a0f856987\") " pod="openshift-ovn-kubernetes/ovnkube-node-lhj9s"
Oct 30 21:53:00.792269 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.792110    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bf0e36d0-87bd-4e1e-be40-8a3a0f856987-run-openvswitch\") pod \"ovnkube-node-lhj9s\" (UID: \"bf0e36d0-87bd-4e1e-be40-8a3a0f856987\") " pod="openshift-ovn-kubernetes/ovnkube-node-lhj9s"
Oct 30 21:53:00.792269 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.792112    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/bf0e36d0-87bd-4e1e-be40-8a3a0f856987-host-kubelet\") pod \"ovnkube-node-lhj9s\" (UID: \"bf0e36d0-87bd-4e1e-be40-8a3a0f856987\") " pod="openshift-ovn-kubernetes/ovnkube-node-lhj9s"
Oct 30 21:53:00.792269 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.792160    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c590d5fe-57de-4905-90fe-8527b215ce94-tuning-conf-dir\") pod \"multus-additional-cni-plugins-vpbvr\" (UID: \"c590d5fe-57de-4905-90fe-8527b215ce94\") " pod="openshift-multus/multus-additional-cni-plugins-vpbvr"
Oct 30 21:53:00.792269 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.792166    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/bf0e36d0-87bd-4e1e-be40-8a3a0f856987-node-log\") pod \"ovnkube-node-lhj9s\" (UID: \"bf0e36d0-87bd-4e1e-be40-8a3a0f856987\") " pod="openshift-ovn-kubernetes/ovnkube-node-lhj9s"
Oct 30 21:53:00.792269 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.792179    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/bf0e36d0-87bd-4e1e-be40-8a3a0f856987-host-kubelet\") pod \"ovnkube-node-lhj9s\" (UID: \"bf0e36d0-87bd-4e1e-be40-8a3a0f856987\") " pod="openshift-ovn-kubernetes/ovnkube-node-lhj9s"
Oct 30 21:53:00.792269 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.792186    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/bf0e36d0-87bd-4e1e-be40-8a3a0f856987-host-slash\") pod \"ovnkube-node-lhj9s\" (UID: \"bf0e36d0-87bd-4e1e-be40-8a3a0f856987\") " pod="openshift-ovn-kubernetes/ovnkube-node-lhj9s"
Oct 30 21:53:00.792269 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.792215    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/bf0e36d0-87bd-4e1e-be40-8a3a0f856987-host-slash\") pod \"ovnkube-node-lhj9s\" (UID: \"bf0e36d0-87bd-4e1e-be40-8a3a0f856987\") " pod="openshift-ovn-kubernetes/ovnkube-node-lhj9s"
Oct 30 21:53:00.792269 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.792217    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c590d5fe-57de-4905-90fe-8527b215ce94-tuning-conf-dir\") pod \"multus-additional-cni-plugins-vpbvr\" (UID: \"c590d5fe-57de-4905-90fe-8527b215ce94\") " pod="openshift-multus/multus-additional-cni-plugins-vpbvr"
Oct 30 21:53:00.792269 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.792211    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/bf0e36d0-87bd-4e1e-be40-8a3a0f856987-run-ovn\") pod \"ovnkube-node-lhj9s\" (UID: \"bf0e36d0-87bd-4e1e-be40-8a3a0f856987\") " pod="openshift-ovn-kubernetes/ovnkube-node-lhj9s"
Oct 30 21:53:00.792269 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.792251    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b1a02001-9121-47bd-ae96-b39196ad2357-host-run-netns\") pod \"multus-hf5t4\" (UID: \"b1a02001-9121-47bd-ae96-b39196ad2357\") " pod="openshift-multus/multus-hf5t4"
Oct 30 21:53:00.792269 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.792260    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/bf0e36d0-87bd-4e1e-be40-8a3a0f856987-run-ovn\") pod \"ovnkube-node-lhj9s\" (UID: \"bf0e36d0-87bd-4e1e-be40-8a3a0f856987\") " pod="openshift-ovn-kubernetes/ovnkube-node-lhj9s"
Oct 30 21:53:00.792269 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.792265    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4a03a836-506f-404e-9c55-c26d779cd4ee-metrics-tls\") pod \"dns-default-8qxwh\" (UID: \"4a03a836-506f-404e-9c55-c26d779cd4ee\") " pod="openshift-dns/dns-default-8qxwh"
Oct 30 21:53:00.792269 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.792280    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/982efb96-3613-4ef2-ac51-ed3f40745a32-host\") pod \"tuned-9zvbw\" (UID: \"982efb96-3613-4ef2-ac51-ed3f40745a32\") " pod="openshift-cluster-node-tuning-operator/tuned-9zvbw"
Oct 30 21:53:00.793047 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.792302    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b1a02001-9121-47bd-ae96-b39196ad2357-host-run-netns\") pod \"multus-hf5t4\" (UID: \"b1a02001-9121-47bd-ae96-b39196ad2357\") " pod="openshift-multus/multus-hf5t4"
Oct 30 21:53:00.793047 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.792303    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/d17b7754-d1d4-4952-97c7-7a04c714b48c-dbus\") pod \"global-pull-secret-syncer-4vtdc\" (UID: \"d17b7754-d1d4-4952-97c7-7a04c714b48c\") " pod="kube-system/global-pull-secret-syncer-4vtdc"
Oct 30 21:53:00.793047 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.792328    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/386f9569-a5f5-4a9e-beee-3005994186cd-socket-dir\") pod \"aws-ebs-csi-driver-node-nd9qk\" (UID: \"386f9569-a5f5-4a9e-beee-3005994186cd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nd9qk"
Oct 30 21:53:00.793047 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.792342    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/982efb96-3613-4ef2-ac51-ed3f40745a32-host\") pod \"tuned-9zvbw\" (UID: \"982efb96-3613-4ef2-ac51-ed3f40745a32\") " pod="openshift-cluster-node-tuning-operator/tuned-9zvbw"
Oct 30 21:53:00.793047 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.792355    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/bf0e36d0-87bd-4e1e-be40-8a3a0f856987-host-cni-bin\") pod \"ovnkube-node-lhj9s\" (UID: \"bf0e36d0-87bd-4e1e-be40-8a3a0f856987\") " pod="openshift-ovn-kubernetes/ovnkube-node-lhj9s"
Oct 30 21:53:00.793047 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.792380    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/982efb96-3613-4ef2-ac51-ed3f40745a32-etc-kubernetes\") pod \"tuned-9zvbw\" (UID: \"982efb96-3613-4ef2-ac51-ed3f40745a32\") " pod="openshift-cluster-node-tuning-operator/tuned-9zvbw"
Oct 30 21:53:00.793047 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.792384    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/d17b7754-d1d4-4952-97c7-7a04c714b48c-dbus\") pod \"global-pull-secret-syncer-4vtdc\" (UID: \"d17b7754-d1d4-4952-97c7-7a04c714b48c\") " pod="kube-system/global-pull-secret-syncer-4vtdc"
Oct 30 21:53:00.793047 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.792404    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/d17b7754-d1d4-4952-97c7-7a04c714b48c-kubelet-config\") pod \"global-pull-secret-syncer-4vtdc\" (UID: \"d17b7754-d1d4-4952-97c7-7a04c714b48c\") " pod="kube-system/global-pull-secret-syncer-4vtdc"
Oct 30 21:53:00.793047 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.792410    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/bf0e36d0-87bd-4e1e-be40-8a3a0f856987-host-cni-bin\") pod \"ovnkube-node-lhj9s\" (UID: \"bf0e36d0-87bd-4e1e-be40-8a3a0f856987\") " pod="openshift-ovn-kubernetes/ovnkube-node-lhj9s"
Oct 30 21:53:00.793047 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.792415    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/386f9569-a5f5-4a9e-beee-3005994186cd-socket-dir\") pod \"aws-ebs-csi-driver-node-nd9qk\" (UID: \"386f9569-a5f5-4a9e-beee-3005994186cd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nd9qk"
Oct 30 21:53:00.793047 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.792433    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/db073c3b-7cee-4e22-8c11-209bbeb21a96-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-bkhmz\" (UID: \"db073c3b-7cee-4e22-8c11-209bbeb21a96\") " pod="openshift-insights/insights-runtime-extractor-bkhmz"
Oct 30 21:53:00.793047 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.792437    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/982efb96-3613-4ef2-ac51-ed3f40745a32-etc-kubernetes\") pod \"tuned-9zvbw\" (UID: \"982efb96-3613-4ef2-ac51-ed3f40745a32\") " pod="openshift-cluster-node-tuning-operator/tuned-9zvbw"
Oct 30 21:53:00.793047 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.792452    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/164bafd8-ff17-46a2-afc0-dd01fa1aaeb6-host\") pod \"node-ca-f9t6t\" (UID: \"164bafd8-ff17-46a2-afc0-dd01fa1aaeb6\") " pod="openshift-image-registry/node-ca-f9t6t"
Oct 30 21:53:00.793047 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.792476    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/164bafd8-ff17-46a2-afc0-dd01fa1aaeb6-host\") pod \"node-ca-f9t6t\" (UID: \"164bafd8-ff17-46a2-afc0-dd01fa1aaeb6\") " pod="openshift-image-registry/node-ca-f9t6t"
Oct 30 21:53:00.793047 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.792479    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a4ff4cce-671b-4136-8b63-b22148c0db4e-host-slash\") pod \"iptables-alerter-2hzq7\" (UID: \"a4ff4cce-671b-4136-8b63-b22148c0db4e\") " pod="openshift-network-operator/iptables-alerter-2hzq7"
Oct 30 21:53:00.793047 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.792501    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/d17b7754-d1d4-4952-97c7-7a04c714b48c-kubelet-config\") pod \"global-pull-secret-syncer-4vtdc\" (UID: \"d17b7754-d1d4-4952-97c7-7a04c714b48c\") " pod="kube-system/global-pull-secret-syncer-4vtdc"
Oct 30 21:53:00.793047 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.792508    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c590d5fe-57de-4905-90fe-8527b215ce94-cnibin\") pod \"multus-additional-cni-plugins-vpbvr\" (UID: \"c590d5fe-57de-4905-90fe-8527b215ce94\") " pod="openshift-multus/multus-additional-cni-plugins-vpbvr"
Oct 30 21:53:00.793047 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.792535    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b1a02001-9121-47bd-ae96-b39196ad2357-cnibin\") pod \"multus-hf5t4\" (UID: \"b1a02001-9121-47bd-ae96-b39196ad2357\") " pod="openshift-multus/multus-hf5t4"
Oct 30 21:53:00.793047 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.792561    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/b1a02001-9121-47bd-ae96-b39196ad2357-hostroot\") pod \"multus-hf5t4\" (UID: \"b1a02001-9121-47bd-ae96-b39196ad2357\") " pod="openshift-multus/multus-hf5t4"
Oct 30 21:53:00.793047 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.792585    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b1a02001-9121-47bd-ae96-b39196ad2357-multus-conf-dir\") pod \"multus-hf5t4\" (UID: \"b1a02001-9121-47bd-ae96-b39196ad2357\") " pod="openshift-multus/multus-hf5t4"
Oct 30 21:53:00.793047 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.792512    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a4ff4cce-671b-4136-8b63-b22148c0db4e-host-slash\") pod \"iptables-alerter-2hzq7\" (UID: \"a4ff4cce-671b-4136-8b63-b22148c0db4e\") " pod="openshift-network-operator/iptables-alerter-2hzq7"
Oct 30 21:53:00.793047 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.792594    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c590d5fe-57de-4905-90fe-8527b215ce94-cnibin\") pod \"multus-additional-cni-plugins-vpbvr\" (UID: \"c590d5fe-57de-4905-90fe-8527b215ce94\") " pod="openshift-multus/multus-additional-cni-plugins-vpbvr"
Oct 30 21:53:00.793047 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.792606    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/b1a02001-9121-47bd-ae96-b39196ad2357-hostroot\") pod \"multus-hf5t4\" (UID: \"b1a02001-9121-47bd-ae96-b39196ad2357\") " pod="openshift-multus/multus-hf5t4"
Oct 30 21:53:00.793047 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.792610    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/982efb96-3613-4ef2-ac51-ed3f40745a32-etc-sysconfig\") pod \"tuned-9zvbw\" (UID: \"982efb96-3613-4ef2-ac51-ed3f40745a32\") " pod="openshift-cluster-node-tuning-operator/tuned-9zvbw"
Oct 30 21:53:00.793047 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.792624    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b1a02001-9121-47bd-ae96-b39196ad2357-cnibin\") pod \"multus-hf5t4\" (UID: \"b1a02001-9121-47bd-ae96-b39196ad2357\") " pod="openshift-multus/multus-hf5t4"
Oct 30 21:53:00.793047 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.792628    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b1a02001-9121-47bd-ae96-b39196ad2357-multus-conf-dir\") pod \"multus-hf5t4\" (UID: \"b1a02001-9121-47bd-ae96-b39196ad2357\") " pod="openshift-multus/multus-hf5t4"
Oct 30 21:53:00.793047 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.792642    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bf0e36d0-87bd-4e1e-be40-8a3a0f856987-var-lib-openvswitch\") pod \"ovnkube-node-lhj9s\" (UID: \"bf0e36d0-87bd-4e1e-be40-8a3a0f856987\") " pod="openshift-ovn-kubernetes/ovnkube-node-lhj9s"
Oct 30 21:53:00.793047 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.792649    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/982efb96-3613-4ef2-ac51-ed3f40745a32-etc-sysconfig\") pod \"tuned-9zvbw\" (UID: \"982efb96-3613-4ef2-ac51-ed3f40745a32\") " pod="openshift-cluster-node-tuning-operator/tuned-9zvbw"
Oct 30 21:53:00.793047 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.792667    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bf0e36d0-87bd-4e1e-be40-8a3a0f856987-etc-openvswitch\") pod \"ovnkube-node-lhj9s\" (UID: \"bf0e36d0-87bd-4e1e-be40-8a3a0f856987\") " pod="openshift-ovn-kubernetes/ovnkube-node-lhj9s"
Oct 30 21:53:00.793047 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.792680    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bf0e36d0-87bd-4e1e-be40-8a3a0f856987-var-lib-openvswitch\") pod \"ovnkube-node-lhj9s\" (UID: \"bf0e36d0-87bd-4e1e-be40-8a3a0f856987\") " pod="openshift-ovn-kubernetes/ovnkube-node-lhj9s"
Oct 30 21:53:00.793047 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.792701    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b1a02001-9121-47bd-ae96-b39196ad2357-host-var-lib-cni-bin\") pod \"multus-hf5t4\" (UID: \"b1a02001-9121-47bd-ae96-b39196ad2357\") " pod="openshift-multus/multus-hf5t4"
Oct 30 21:53:00.793047 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.792722    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bf0e36d0-87bd-4e1e-be40-8a3a0f856987-etc-openvswitch\") pod \"ovnkube-node-lhj9s\" (UID: \"bf0e36d0-87bd-4e1e-be40-8a3a0f856987\") " pod="openshift-ovn-kubernetes/ovnkube-node-lhj9s"
Oct 30 21:53:00.793047 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.792730    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/982efb96-3613-4ef2-ac51-ed3f40745a32-sys\") pod \"tuned-9zvbw\" (UID: \"982efb96-3613-4ef2-ac51-ed3f40745a32\") " pod="openshift-cluster-node-tuning-operator/tuned-9zvbw"
Oct 30 21:53:00.793047 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.792757    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b1a02001-9121-47bd-ae96-b39196ad2357-host-var-lib-cni-bin\") pod \"multus-hf5t4\" (UID: \"b1a02001-9121-47bd-ae96-b39196ad2357\") " pod="openshift-multus/multus-hf5t4"
Oct 30 21:53:00.793047 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.792776    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/982efb96-3613-4ef2-ac51-ed3f40745a32-var-lib-kubelet\") pod \"tuned-9zvbw\" (UID: \"982efb96-3613-4ef2-ac51-ed3f40745a32\") " pod="openshift-cluster-node-tuning-operator/tuned-9zvbw"
Oct 30 21:53:00.793047 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.792784    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/982efb96-3613-4ef2-ac51-ed3f40745a32-sys\") pod \"tuned-9zvbw\" (UID: \"982efb96-3613-4ef2-ac51-ed3f40745a32\") " pod="openshift-cluster-node-tuning-operator/tuned-9zvbw"
Oct 30 21:53:00.793047 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.792792    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/982efb96-3613-4ef2-ac51-ed3f40745a32-etc-sysctl-conf\") pod \"tuned-9zvbw\" (UID: \"982efb96-3613-4ef2-ac51-ed3f40745a32\") " pod="openshift-cluster-node-tuning-operator/tuned-9zvbw"
Oct 30 21:53:00.793047 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.792819    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b1a02001-9121-47bd-ae96-b39196ad2357-multus-cni-dir\") pod \"multus-hf5t4\" (UID: \"b1a02001-9121-47bd-ae96-b39196ad2357\") " pod="openshift-multus/multus-hf5t4"
Oct 30 21:53:00.793047 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.792833    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/982efb96-3613-4ef2-ac51-ed3f40745a32-var-lib-kubelet\") pod \"tuned-9zvbw\" (UID: \"982efb96-3613-4ef2-ac51-ed3f40745a32\") " pod="openshift-cluster-node-tuning-operator/tuned-9zvbw"
Oct 30 21:53:00.793047 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.792853    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/b1a02001-9121-47bd-ae96-b39196ad2357-host-var-lib-cni-multus\") pod \"multus-hf5t4\" (UID: \"b1a02001-9121-47bd-ae96-b39196ad2357\") " pod="openshift-multus/multus-hf5t4"
Oct 30 21:53:00.793047 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.792875    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bf0e36d0-87bd-4e1e-be40-8a3a0f856987-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-lhj9s\" (UID: \"bf0e36d0-87bd-4e1e-be40-8a3a0f856987\") " pod="openshift-ovn-kubernetes/ovnkube-node-lhj9s"
Oct 30 21:53:00.793047 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.792902    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/b1a02001-9121-47bd-ae96-b39196ad2357-host-var-lib-cni-multus\") pod \"multus-hf5t4\" (UID: \"b1a02001-9121-47bd-ae96-b39196ad2357\") " pod="openshift-multus/multus-hf5t4"
Oct 30 21:53:00.793047 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.792900    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/386f9569-a5f5-4a9e-beee-3005994186cd-device-dir\") pod \"aws-ebs-csi-driver-node-nd9qk\" (UID: \"386f9569-a5f5-4a9e-beee-3005994186cd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nd9qk"
Oct 30 21:53:00.793047 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.792910    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b1a02001-9121-47bd-ae96-b39196ad2357-multus-cni-dir\") pod \"multus-hf5t4\" (UID: \"b1a02001-9121-47bd-ae96-b39196ad2357\") " pod="openshift-multus/multus-hf5t4"
Oct 30 21:53:00.793047 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.792915    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bf0e36d0-87bd-4e1e-be40-8a3a0f856987-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-lhj9s\" (UID: \"bf0e36d0-87bd-4e1e-be40-8a3a0f856987\") " pod="openshift-ovn-kubernetes/ovnkube-node-lhj9s"
Oct 30 21:53:00.793047 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.792928    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/982efb96-3613-4ef2-ac51-ed3f40745a32-etc-sysctl-conf\") pod \"tuned-9zvbw\" (UID: \"982efb96-3613-4ef2-ac51-ed3f40745a32\") " pod="openshift-cluster-node-tuning-operator/tuned-9zvbw"
Oct 30 21:53:00.793047 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.792943    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/386f9569-a5f5-4a9e-beee-3005994186cd-device-dir\") pod \"aws-ebs-csi-driver-node-nd9qk\" (UID: \"386f9569-a5f5-4a9e-beee-3005994186cd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nd9qk"
Oct 30 21:53:00.793047 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.792950    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/386f9569-a5f5-4a9e-beee-3005994186cd-sys-fs\") pod \"aws-ebs-csi-driver-node-nd9qk\" (UID: \"386f9569-a5f5-4a9e-beee-3005994186cd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nd9qk"
Oct 30 21:53:00.793047 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.792968    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/bf0e36d0-87bd-4e1e-be40-8a3a0f856987-host-run-netns\") pod \"ovnkube-node-lhj9s\" (UID: \"bf0e36d0-87bd-4e1e-be40-8a3a0f856987\") " pod="openshift-ovn-kubernetes/ovnkube-node-lhj9s"
Oct 30 21:53:00.793047 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.792983    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/dffd7a70-b7f3-4e1b-b3ab-74c5a574cab2-hosts-file\") pod \"node-resolver-7v7fb\" (UID: \"dffd7a70-b7f3-4e1b-b3ab-74c5a574cab2\") " pod="openshift-dns/node-resolver-7v7fb"
Oct 30 21:53:00.793047 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.792998    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8806f1e1-0057-4738-8fbc-7e503e1f91ca-metrics-certs\") pod \"network-metrics-daemon-wpdr9\" (UID: \"8806f1e1-0057-4738-8fbc-7e503e1f91ca\") " pod="openshift-multus/network-metrics-daemon-wpdr9"
Oct 30 21:53:00.793047 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.793013    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c590d5fe-57de-4905-90fe-8527b215ce94-system-cni-dir\") pod \"multus-additional-cni-plugins-vpbvr\" (UID: \"c590d5fe-57de-4905-90fe-8527b215ce94\") " pod="openshift-multus/multus-additional-cni-plugins-vpbvr"
Oct 30 21:53:00.793047 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.793027    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bf0e36d0-87bd-4e1e-be40-8a3a0f856987-host-run-ovn-kubernetes\") pod \"ovnkube-node-lhj9s\" (UID: \"bf0e36d0-87bd-4e1e-be40-8a3a0f856987\") " pod="openshift-ovn-kubernetes/ovnkube-node-lhj9s"
Oct 30 21:53:00.793047 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.793033    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/bf0e36d0-87bd-4e1e-be40-8a3a0f856987-host-run-netns\") pod \"ovnkube-node-lhj9s\" (UID: \"bf0e36d0-87bd-4e1e-be40-8a3a0f856987\") " pod="openshift-ovn-kubernetes/ovnkube-node-lhj9s"
Oct 30 21:53:00.793047 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.793043    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/bf0e36d0-87bd-4e1e-be40-8a3a0f856987-host-cni-netd\") pod \"ovnkube-node-lhj9s\" (UID: \"bf0e36d0-87bd-4e1e-be40-8a3a0f856987\") " pod="openshift-ovn-kubernetes/ovnkube-node-lhj9s"
Oct 30 21:53:00.793047 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.793054    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/dffd7a70-b7f3-4e1b-b3ab-74c5a574cab2-hosts-file\") pod \"node-resolver-7v7fb\" (UID: \"dffd7a70-b7f3-4e1b-b3ab-74c5a574cab2\") " pod="openshift-dns/node-resolver-7v7fb"
Oct 30 21:53:00.793047 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.793067    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/b1a02001-9121-47bd-ae96-b39196ad2357-multus-socket-dir-parent\") pod \"multus-hf5t4\" (UID: \"b1a02001-9121-47bd-ae96-b39196ad2357\") " pod="openshift-multus/multus-hf5t4"
Oct 30 21:53:00.793047 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.793077    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/386f9569-a5f5-4a9e-beee-3005994186cd-sys-fs\") pod \"aws-ebs-csi-driver-node-nd9qk\" (UID: \"386f9569-a5f5-4a9e-beee-3005994186cd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nd9qk"
Oct 30 21:53:00.793047 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.793083    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c590d5fe-57de-4905-90fe-8527b215ce94-system-cni-dir\") pod \"multus-additional-cni-plugins-vpbvr\" (UID: \"c590d5fe-57de-4905-90fe-8527b215ce94\") " pod="openshift-multus/multus-additional-cni-plugins-vpbvr"
Oct 30 21:53:00.794752 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.793097    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/b1a02001-9121-47bd-ae96-b39196ad2357-host-run-k8s-cni-cncf-io\") pod \"multus-hf5t4\" (UID: \"b1a02001-9121-47bd-ae96-b39196ad2357\") " pod="openshift-multus/multus-hf5t4"
Oct 30 21:53:00.794752 ip-10-0-131-202 kubenswrapper[4412]: E1030 21:53:00.793104    4412 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Oct 30 21:53:00.794752 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.793139    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/db073c3b-7cee-4e22-8c11-209bbeb21a96-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-bkhmz\" (UID: \"db073c3b-7cee-4e22-8c11-209bbeb21a96\") " pod="openshift-insights/insights-runtime-extractor-bkhmz"
Oct 30 21:53:00.794752 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.793098    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bf0e36d0-87bd-4e1e-be40-8a3a0f856987-host-run-ovn-kubernetes\") pod \"ovnkube-node-lhj9s\" (UID: \"bf0e36d0-87bd-4e1e-be40-8a3a0f856987\") " pod="openshift-ovn-kubernetes/ovnkube-node-lhj9s"
Oct 30 21:53:00.794752 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.793154    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/b1a02001-9121-47bd-ae96-b39196ad2357-multus-socket-dir-parent\") pod \"multus-hf5t4\" (UID: \"b1a02001-9121-47bd-ae96-b39196ad2357\") " pod="openshift-multus/multus-hf5t4"
Oct 30 21:53:00.794752 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.793099    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/bf0e36d0-87bd-4e1e-be40-8a3a0f856987-host-cni-netd\") pod \"ovnkube-node-lhj9s\" (UID: \"bf0e36d0-87bd-4e1e-be40-8a3a0f856987\") " pod="openshift-ovn-kubernetes/ovnkube-node-lhj9s"
Oct 30 21:53:00.794752 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.793155    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/386f9569-a5f5-4a9e-beee-3005994186cd-registration-dir\") pod \"aws-ebs-csi-driver-node-nd9qk\" (UID: \"386f9569-a5f5-4a9e-beee-3005994186cd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nd9qk"
Oct 30 21:53:00.794752 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.793190    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/b1a02001-9121-47bd-ae96-b39196ad2357-host-run-k8s-cni-cncf-io\") pod \"multus-hf5t4\" (UID: \"b1a02001-9121-47bd-ae96-b39196ad2357\") " pod="openshift-multus/multus-hf5t4"
Oct 30 21:53:00.794752 ip-10-0-131-202 kubenswrapper[4412]: E1030 21:53:00.793198    4412 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8806f1e1-0057-4738-8fbc-7e503e1f91ca-metrics-certs podName:8806f1e1-0057-4738-8fbc-7e503e1f91ca nodeName:}" failed. No retries permitted until 2026-10-30 21:53:01.293178698 +0000 UTC m=+1.797850652 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8806f1e1-0057-4738-8fbc-7e503e1f91ca-metrics-certs") pod "network-metrics-daemon-wpdr9" (UID: "8806f1e1-0057-4738-8fbc-7e503e1f91ca") : secret "metrics-daemon-secret" not found
Oct 30 21:53:00.794752 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.793213    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/386f9569-a5f5-4a9e-beee-3005994186cd-registration-dir\") pod \"aws-ebs-csi-driver-node-nd9qk\" (UID: \"386f9569-a5f5-4a9e-beee-3005994186cd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nd9qk"
Oct 30 21:53:00.794752 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.793228    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b1a02001-9121-47bd-ae96-b39196ad2357-host-var-lib-kubelet\") pod \"multus-hf5t4\" (UID: \"b1a02001-9121-47bd-ae96-b39196ad2357\") " pod="openshift-multus/multus-hf5t4"
Oct 30 21:53:00.794752 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.793247    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/b1a02001-9121-47bd-ae96-b39196ad2357-host-run-multus-certs\") pod \"multus-hf5t4\" (UID: \"b1a02001-9121-47bd-ae96-b39196ad2357\") " pod="openshift-multus/multus-hf5t4"
Oct 30 21:53:00.794752 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.793261    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b1a02001-9121-47bd-ae96-b39196ad2357-etc-kubernetes\") pod \"multus-hf5t4\" (UID: \"b1a02001-9121-47bd-ae96-b39196ad2357\") " pod="openshift-multus/multus-hf5t4"
Oct 30 21:53:00.794752 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.793274    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b1a02001-9121-47bd-ae96-b39196ad2357-host-var-lib-kubelet\") pod \"multus-hf5t4\" (UID: \"b1a02001-9121-47bd-ae96-b39196ad2357\") " pod="openshift-multus/multus-hf5t4"
Oct 30 21:53:00.794752 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.793278    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/982efb96-3613-4ef2-ac51-ed3f40745a32-etc-modprobe-d\") pod \"tuned-9zvbw\" (UID: \"982efb96-3613-4ef2-ac51-ed3f40745a32\") " pod="openshift-cluster-node-tuning-operator/tuned-9zvbw"
Oct 30 21:53:00.794752 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.793293    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/982efb96-3613-4ef2-ac51-ed3f40745a32-etc-sysctl-d\") pod \"tuned-9zvbw\" (UID: \"982efb96-3613-4ef2-ac51-ed3f40745a32\") " pod="openshift-cluster-node-tuning-operator/tuned-9zvbw"
Oct 30 21:53:00.794752 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.793296    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/b1a02001-9121-47bd-ae96-b39196ad2357-host-run-multus-certs\") pod \"multus-hf5t4\" (UID: \"b1a02001-9121-47bd-ae96-b39196ad2357\") " pod="openshift-multus/multus-hf5t4"
Oct 30 21:53:00.794752 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.793307    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/982efb96-3613-4ef2-ac51-ed3f40745a32-etc-systemd\") pod \"tuned-9zvbw\" (UID: \"982efb96-3613-4ef2-ac51-ed3f40745a32\") " pod="openshift-cluster-node-tuning-operator/tuned-9zvbw"
Oct 30 21:53:00.794752 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.793301    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b1a02001-9121-47bd-ae96-b39196ad2357-etc-kubernetes\") pod \"multus-hf5t4\" (UID: \"b1a02001-9121-47bd-ae96-b39196ad2357\") " pod="openshift-multus/multus-hf5t4"
Oct 30 21:53:00.794752 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.793346    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/386f9569-a5f5-4a9e-beee-3005994186cd-etc-selinux\") pod \"aws-ebs-csi-driver-node-nd9qk\" (UID: \"386f9569-a5f5-4a9e-beee-3005994186cd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nd9qk"
Oct 30 21:53:00.794752 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.793371    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/982efb96-3613-4ef2-ac51-ed3f40745a32-etc-modprobe-d\") pod \"tuned-9zvbw\" (UID: \"982efb96-3613-4ef2-ac51-ed3f40745a32\") " pod="openshift-cluster-node-tuning-operator/tuned-9zvbw"
Oct 30 21:53:00.794752 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.793347    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/982efb96-3613-4ef2-ac51-ed3f40745a32-etc-systemd\") pod \"tuned-9zvbw\" (UID: \"982efb96-3613-4ef2-ac51-ed3f40745a32\") " pod="openshift-cluster-node-tuning-operator/tuned-9zvbw"
Oct 30 21:53:00.794752 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.793374    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/bf0e36d0-87bd-4e1e-be40-8a3a0f856987-systemd-units\") pod \"ovnkube-node-lhj9s\" (UID: \"bf0e36d0-87bd-4e1e-be40-8a3a0f856987\") " pod="openshift-ovn-kubernetes/ovnkube-node-lhj9s"
Oct 30 21:53:00.794752 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.793381    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/386f9569-a5f5-4a9e-beee-3005994186cd-etc-selinux\") pod \"aws-ebs-csi-driver-node-nd9qk\" (UID: \"386f9569-a5f5-4a9e-beee-3005994186cd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nd9qk"
Oct 30 21:53:00.794752 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.793407    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/982efb96-3613-4ef2-ac51-ed3f40745a32-etc-sysctl-d\") pod \"tuned-9zvbw\" (UID: \"982efb96-3613-4ef2-ac51-ed3f40745a32\") " pod="openshift-cluster-node-tuning-operator/tuned-9zvbw"
Oct 30 21:53:00.794752 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.793412    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/bf0e36d0-87bd-4e1e-be40-8a3a0f856987-run-systemd\") pod \"ovnkube-node-lhj9s\" (UID: \"bf0e36d0-87bd-4e1e-be40-8a3a0f856987\") " pod="openshift-ovn-kubernetes/ovnkube-node-lhj9s"
Oct 30 21:53:00.794752 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.793436    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/bf0e36d0-87bd-4e1e-be40-8a3a0f856987-run-systemd\") pod \"ovnkube-node-lhj9s\" (UID: \"bf0e36d0-87bd-4e1e-be40-8a3a0f856987\") " pod="openshift-ovn-kubernetes/ovnkube-node-lhj9s"
Oct 30 21:53:00.794752 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.793406    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/bf0e36d0-87bd-4e1e-be40-8a3a0f856987-systemd-units\") pod \"ovnkube-node-lhj9s\" (UID: \"bf0e36d0-87bd-4e1e-be40-8a3a0f856987\") " pod="openshift-ovn-kubernetes/ovnkube-node-lhj9s"
Oct 30 21:53:00.794752 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.793436    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/db073c3b-7cee-4e22-8c11-209bbeb21a96-crio-socket\") pod \"insights-runtime-extractor-bkhmz\" (UID: \"db073c3b-7cee-4e22-8c11-209bbeb21a96\") " pod="openshift-insights/insights-runtime-extractor-bkhmz"
Oct 30 21:53:00.794752 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.793465    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/db073c3b-7cee-4e22-8c11-209bbeb21a96-crio-socket\") pod \"insights-runtime-extractor-bkhmz\" (UID: \"db073c3b-7cee-4e22-8c11-209bbeb21a96\") " pod="openshift-insights/insights-runtime-extractor-bkhmz"
Oct 30 21:53:00.794752 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.793468    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/386f9569-a5f5-4a9e-beee-3005994186cd-kubelet-dir\") pod \"aws-ebs-csi-driver-node-nd9qk\" (UID: \"386f9569-a5f5-4a9e-beee-3005994186cd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nd9qk"
Oct 30 21:53:00.794752 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.793496    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/386f9569-a5f5-4a9e-beee-3005994186cd-kubelet-dir\") pod \"aws-ebs-csi-driver-node-nd9qk\" (UID: \"386f9569-a5f5-4a9e-beee-3005994186cd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nd9qk"
Oct 30 21:53:00.830343 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.830282    4412 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Oct 30 21:53:00.830343 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.830329    4412 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Oct 30 21:53:00.830506 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.830362    4412 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Oct 30 21:53:00.830506 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.830373    4412 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Oct 30 21:53:00.830506 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.830431    4412 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Oct 30 21:53:00.830506 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.830434    4412 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Oct 30 21:53:00.830506 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.830481    4412 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Oct 30 21:53:00.830506 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.830483    4412 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Oct 30 21:53:00.830751 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.830635    4412 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Oct 30 21:53:00.830751 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.830644    4412 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Oct 30 21:53:00.830751 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.830674    4412 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Oct 30 21:53:00.830890 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.830877    4412 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Oct 30 21:53:00.830932 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.830912    4412 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Oct 30 21:53:00.830984 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.830935    4412 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Oct 30 21:53:00.830984 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.830939    4412 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Oct 30 21:53:00.830984 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.830951    4412 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Oct 30 21:53:00.830984 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.830966    4412 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Oct 30 21:53:00.830984 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.830973    4412 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Oct 30 21:53:00.830984 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.830973    4412 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Oct 30 21:53:00.831302 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.831076    4412 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Oct 30 21:53:00.831302 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.831085    4412 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Oct 30 21:53:00.831302 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.831134    4412 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\""
Oct 30 21:53:00.831302 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.831167    4412 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Oct 30 21:53:00.831302 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.831167    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/b1a02001-9121-47bd-ae96-b39196ad2357-multus-daemon-config\") pod \"multus-hf5t4\" (UID: \"b1a02001-9121-47bd-ae96-b39196ad2357\") " pod="openshift-multus/multus-hf5t4"
Oct 30 21:53:00.831302 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.831202    4412 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Oct 30 21:53:00.831302 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.831184    4412 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Oct 30 21:53:00.831302 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.831222    4412 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Oct 30 21:53:00.831302 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.831232    4412 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Oct 30 21:53:00.831635 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.831345    4412 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\""
Oct 30 21:53:00.831635 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.831358    4412 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Oct 30 21:53:00.831635 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.831312    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/4a03a836-506f-404e-9c55-c26d779cd4ee-config-volume\") pod \"dns-default-8qxwh\" (UID: \"4a03a836-506f-404e-9c55-c26d779cd4ee\") " pod="openshift-dns/dns-default-8qxwh"
Oct 30 21:53:00.831635 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.831384    4412 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Oct 30 21:53:00.831635 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.831428    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c590d5fe-57de-4905-90fe-8527b215ce94-cni-binary-copy\") pod \"multus-additional-cni-plugins-vpbvr\" (UID: \"c590d5fe-57de-4905-90fe-8527b215ce94\") " pod="openshift-multus/multus-additional-cni-plugins-vpbvr"
Oct 30 21:53:00.831635 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.831451    4412 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Oct 30 21:53:00.831635 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.831493    4412 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Oct 30 21:53:00.831635 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.831500    4412 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Oct 30 21:53:00.832684 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.832478    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/25590e1f-99e3-4469-8a69-39cd30317c28-konnectivity-ca\") pod \"konnectivity-agent-x7l5l\" (UID: \"25590e1f-99e3-4469-8a69-39cd30317c28\") " pod="kube-system/konnectivity-agent-x7l5l"
Oct 30 21:53:00.832770 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.832686    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vd7d5\" (UniqueName: \"kubernetes.io/projected/db073c3b-7cee-4e22-8c11-209bbeb21a96-kube-api-access-vd7d5\") pod \"insights-runtime-extractor-bkhmz\" (UID: \"db073c3b-7cee-4e22-8c11-209bbeb21a96\") " pod="openshift-insights/insights-runtime-extractor-bkhmz"
Oct 30 21:53:00.832770 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.832691    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fwsxm\" (UniqueName: \"kubernetes.io/projected/8806f1e1-0057-4738-8fbc-7e503e1f91ca-kube-api-access-fwsxm\") pod \"network-metrics-daemon-wpdr9\" (UID: \"8806f1e1-0057-4738-8fbc-7e503e1f91ca\") " pod="openshift-multus/network-metrics-daemon-wpdr9"
Oct 30 21:53:00.832893 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.832871    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2pwzh\" (UniqueName: \"kubernetes.io/projected/982efb96-3613-4ef2-ac51-ed3f40745a32-kube-api-access-2pwzh\") pod \"tuned-9zvbw\" (UID: \"982efb96-3613-4ef2-ac51-ed3f40745a32\") " pod="openshift-cluster-node-tuning-operator/tuned-9zvbw"
Oct 30 21:53:00.832948 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.832920    4412 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-check-target-dockercfg-gmfbd\""
Oct 30 21:53:00.832997 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.832953    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fq868\" (UniqueName: \"kubernetes.io/projected/c590d5fe-57de-4905-90fe-8527b215ce94-kube-api-access-fq868\") pod \"multus-additional-cni-plugins-vpbvr\" (UID: \"c590d5fe-57de-4905-90fe-8527b215ce94\") " pod="openshift-multus/multus-additional-cni-plugins-vpbvr"
Oct 30 21:53:00.832997 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.832964    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dtsnb\" (UniqueName: \"kubernetes.io/projected/4a03a836-506f-404e-9c55-c26d779cd4ee-kube-api-access-dtsnb\") pod \"dns-default-8qxwh\" (UID: \"4a03a836-506f-404e-9c55-c26d779cd4ee\") " pod="openshift-dns/dns-default-8qxwh"
Oct 30 21:53:00.833097 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.833004    4412 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Oct 30 21:53:00.833097 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.833060    4412 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Oct 30 21:53:00.833215 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.833149    4412 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-hqgxz\""
Oct 30 21:53:00.833215 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.833156    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hrlj5\" (UniqueName: \"kubernetes.io/projected/dffd7a70-b7f3-4e1b-b3ab-74c5a574cab2-kube-api-access-hrlj5\") pod \"node-resolver-7v7fb\" (UID: \"dffd7a70-b7f3-4e1b-b3ab-74c5a574cab2\") " pod="openshift-dns/node-resolver-7v7fb"
Oct 30 21:53:00.833215 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.833208    4412 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-node-identity-dockercfg-gqgtq\""
Oct 30 21:53:00.833315 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.833187    4412 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent-dockercfg-ppl5h\""
Oct 30 21:53:00.833315 ip-10-0-131-202 kubenswrapper[4412]: E1030 21:53:00.833281    4412 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Oct 30 21:53:00.833372 ip-10-0-131-202 kubenswrapper[4412]: E1030 21:53:00.833368    4412 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/db073c3b-7cee-4e22-8c11-209bbeb21a96-kube-rbac-proxy-cm podName:db073c3b-7cee-4e22-8c11-209bbeb21a96 nodeName:}" failed. No retries permitted until 2026-10-30 21:53:01.333355675 +0000 UTC m=+1.838027605 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/db073c3b-7cee-4e22-8c11-209bbeb21a96-kube-rbac-proxy-cm") pod "insights-runtime-extractor-bkhmz" (UID: "db073c3b-7cee-4e22-8c11-209bbeb21a96") : configmap "kube-rbac-proxy" not found
Oct 30 21:53:00.834021 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.834008    4412 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-ww8ls\""
Oct 30 21:53:00.838451 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.838436    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6r8d9\" (UniqueName: \"kubernetes.io/projected/a4ff4cce-671b-4136-8b63-b22148c0db4e-kube-api-access-6r8d9\") pod \"iptables-alerter-2hzq7\" (UID: \"a4ff4cce-671b-4136-8b63-b22148c0db4e\") " pod="openshift-network-operator/iptables-alerter-2hzq7"
Oct 30 21:53:00.839388 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.839333    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/bf0e36d0-87bd-4e1e-be40-8a3a0f856987-env-overrides\") pod \"ovnkube-node-lhj9s\" (UID: \"bf0e36d0-87bd-4e1e-be40-8a3a0f856987\") " pod="openshift-ovn-kubernetes/ovnkube-node-lhj9s"
Oct 30 21:53:00.839494 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.839385    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/a4ff4cce-671b-4136-8b63-b22148c0db4e-iptables-alerter-script\") pod \"iptables-alerter-2hzq7\" (UID: \"a4ff4cce-671b-4136-8b63-b22148c0db4e\") " pod="openshift-network-operator/iptables-alerter-2hzq7"
Oct 30 21:53:00.839494 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.839421    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/bf0e36d0-87bd-4e1e-be40-8a3a0f856987-ovnkube-script-lib\") pod \"ovnkube-node-lhj9s\" (UID: \"bf0e36d0-87bd-4e1e-be40-8a3a0f856987\") " pod="openshift-ovn-kubernetes/ovnkube-node-lhj9s"
Oct 30 21:53:00.839617 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.839580    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/164bafd8-ff17-46a2-afc0-dd01fa1aaeb6-serviceca\") pod \"node-ca-f9t6t\" (UID: \"164bafd8-ff17-46a2-afc0-dd01fa1aaeb6\") " pod="openshift-image-registry/node-ca-f9t6t"
Oct 30 21:53:00.839617 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.839565    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b1a02001-9121-47bd-ae96-b39196ad2357-cni-binary-copy\") pod \"multus-hf5t4\" (UID: \"b1a02001-9121-47bd-ae96-b39196ad2357\") " pod="openshift-multus/multus-hf5t4"
Oct 30 21:53:00.839718 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.839681    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-78t6j\" (UniqueName: \"kubernetes.io/projected/164bafd8-ff17-46a2-afc0-dd01fa1aaeb6-kube-api-access-78t6j\") pod \"node-ca-f9t6t\" (UID: \"164bafd8-ff17-46a2-afc0-dd01fa1aaeb6\") " pod="openshift-image-registry/node-ca-f9t6t"
Oct 30 21:53:00.839802 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.839780    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tg4f9\" (UniqueName: \"kubernetes.io/projected/386f9569-a5f5-4a9e-beee-3005994186cd-kube-api-access-tg4f9\") pod \"aws-ebs-csi-driver-node-nd9qk\" (UID: \"386f9569-a5f5-4a9e-beee-3005994186cd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nd9qk"
Oct 30 21:53:00.839906 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.839835    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ctg2b\" (UniqueName: \"kubernetes.io/projected/bf0e36d0-87bd-4e1e-be40-8a3a0f856987-kube-api-access-ctg2b\") pod \"ovnkube-node-lhj9s\" (UID: \"bf0e36d0-87bd-4e1e-be40-8a3a0f856987\") " pod="openshift-ovn-kubernetes/ovnkube-node-lhj9s"
Oct 30 21:53:00.839906 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.839900    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fsf4l\" (UniqueName: \"kubernetes.io/projected/7e6b0df0-07eb-4301-adaa-1a6514639d9c-kube-api-access-fsf4l\") pod \"migrator-66df5d496b-zbfpb\" (UID: \"7e6b0df0-07eb-4301-adaa-1a6514639d9c\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-zbfpb"
Oct 30 21:53:00.840006 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.839946    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6j99v\" (UniqueName: \"kubernetes.io/projected/7be00c31-281f-4d1c-8e12-1bf74bbe7cbd-kube-api-access-6j99v\") pod \"network-check-target-slfvl\" (UID: \"7be00c31-281f-4d1c-8e12-1bf74bbe7cbd\") " pod="openshift-network-diagnostics/network-check-target-slfvl"
Oct 30 21:53:00.840106 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.840092    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/c590d5fe-57de-4905-90fe-8527b215ce94-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-vpbvr\" (UID: \"c590d5fe-57de-4905-90fe-8527b215ce94\") " pod="openshift-multus/multus-additional-cni-plugins-vpbvr"
Oct 30 21:53:00.840200 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.840186    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vsg28\" (UniqueName: \"kubernetes.io/projected/b1a02001-9121-47bd-ae96-b39196ad2357-kube-api-access-vsg28\") pod \"multus-hf5t4\" (UID: \"b1a02001-9121-47bd-ae96-b39196ad2357\") " pod="openshift-multus/multus-hf5t4"
Oct 30 21:53:00.840233 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.840187    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c622t\" (UniqueName: \"kubernetes.io/projected/2345ff5b-db97-4855-8adb-59b9a780615e-kube-api-access-c622t\") pod \"ingress-canary-h2pjp\" (UID: \"2345ff5b-db97-4855-8adb-59b9a780615e\") " pod="openshift-ingress-canary/ingress-canary-h2pjp"
Oct 30 21:53:00.840811 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.840797    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/c590d5fe-57de-4905-90fe-8527b215ce94-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-vpbvr\" (UID: \"c590d5fe-57de-4905-90fe-8527b215ce94\") " pod="openshift-multus/multus-additional-cni-plugins-vpbvr"
Oct 30 21:53:00.840890 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.840877    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/bf0e36d0-87bd-4e1e-be40-8a3a0f856987-ovnkube-config\") pod \"ovnkube-node-lhj9s\" (UID: \"bf0e36d0-87bd-4e1e-be40-8a3a0f856987\") " pod="openshift-ovn-kubernetes/ovnkube-node-lhj9s"
Oct 30 21:53:00.841275 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.841262    4412 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"ingress-canary-dockercfg-qt896\""
Oct 30 21:53:00.843364 ip-10-0-131-202 kubenswrapper[4412]: E1030 21:53:00.843352    4412 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Oct 30 21:53:00.843417 ip-10-0-131-202 kubenswrapper[4412]: E1030 21:53:00.843401    4412 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/db073c3b-7cee-4e22-8c11-209bbeb21a96-insights-runtime-extractor-tls podName:db073c3b-7cee-4e22-8c11-209bbeb21a96 nodeName:}" failed. No retries permitted until 2026-10-30 21:53:01.343389069 +0000 UTC m=+1.848061003 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/db073c3b-7cee-4e22-8c11-209bbeb21a96-insights-runtime-extractor-tls") pod "insights-runtime-extractor-bkhmz" (UID: "db073c3b-7cee-4e22-8c11-209bbeb21a96") : secret "insights-runtime-extractor-tls" not found
Oct 30 21:53:00.843464 ip-10-0-131-202 kubenswrapper[4412]: E1030 21:53:00.843420    4412 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Oct 30 21:53:00.843464 ip-10-0-131-202 kubenswrapper[4412]: E1030 21:53:00.843440    4412 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4a03a836-506f-404e-9c55-c26d779cd4ee-metrics-tls podName:4a03a836-506f-404e-9c55-c26d779cd4ee nodeName:}" failed. No retries permitted until 2026-10-30 21:53:01.343434528 +0000 UTC m=+1.848106461 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4a03a836-506f-404e-9c55-c26d779cd4ee-metrics-tls") pod "dns-default-8qxwh" (UID: "4a03a836-506f-404e-9c55-c26d779cd4ee") : secret "dns-default-metrics-tls" not found
Oct 30 21:53:00.862208 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.862190    4412 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Oct 30 21:53:00.862438 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.862386    4412 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-mgstz\""
Oct 30 21:53:00.862590 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.862489    4412 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-h6mh8\""
Oct 30 21:53:00.862654 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.862615    4412 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Oct 30 21:53:00.862706 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.862685    4412 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-bdqch\""
Oct 30 21:53:00.862795 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.862780    4412 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Oct 30 21:53:00.862843 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.862812    4412 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-7bphz\""
Oct 30 21:53:00.862900 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.862881    4412 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-jfz4q\""
Oct 30 21:53:00.863029 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.862886    4412 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"global-pull-secret-syncer-dockercfg-6q7ft\""
Oct 30 21:53:00.863029 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.862891    4412 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-gsd79\""
Oct 30 21:53:00.863029 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.862617    4412 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-vk8r7\""
Oct 30 21:53:00.863217 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.863204    4412 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-gg6dx\""
Oct 30 21:53:00.863301 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.863290    4412 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Oct 30 21:53:00.869931 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.869915    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d17b7754-d1d4-4952-97c7-7a04c714b48c-original-pull-secret\") pod \"global-pull-secret-syncer-4vtdc\" (UID: \"d17b7754-d1d4-4952-97c7-7a04c714b48c\") " pod="kube-system/global-pull-secret-syncer-4vtdc"
Oct 30 21:53:00.871052 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.871030    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/bf0e36d0-87bd-4e1e-be40-8a3a0f856987-ovn-node-metrics-cert\") pod \"ovnkube-node-lhj9s\" (UID: \"bf0e36d0-87bd-4e1e-be40-8a3a0f856987\") " pod="openshift-ovn-kubernetes/ovnkube-node-lhj9s"
Oct 30 21:53:00.871142 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.871047    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/25590e1f-99e3-4469-8a69-39cd30317c28-agent-certs\") pod \"konnectivity-agent-x7l5l\" (UID: \"25590e1f-99e3-4469-8a69-39cd30317c28\") " pod="kube-system/konnectivity-agent-x7l5l"
Oct 30 21:53:00.871142 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.871103    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2345ff5b-db97-4855-8adb-59b9a780615e-cert\") pod \"ingress-canary-h2pjp\" (UID: \"2345ff5b-db97-4855-8adb-59b9a780615e\") " pod="openshift-ingress-canary/ingress-canary-h2pjp"
Oct 30 21:53:00.984695 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:00.984676    4412 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 21:53:01.297988 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:01.297954    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8806f1e1-0057-4738-8fbc-7e503e1f91ca-metrics-certs\") pod \"network-metrics-daemon-wpdr9\" (UID: \"8806f1e1-0057-4738-8fbc-7e503e1f91ca\") " pod="openshift-multus/network-metrics-daemon-wpdr9"
Oct 30 21:53:01.298170 ip-10-0-131-202 kubenswrapper[4412]: E1030 21:53:01.298073    4412 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Oct 30 21:53:01.298170 ip-10-0-131-202 kubenswrapper[4412]: E1030 21:53:01.298155    4412 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8806f1e1-0057-4738-8fbc-7e503e1f91ca-metrics-certs podName:8806f1e1-0057-4738-8fbc-7e503e1f91ca nodeName:}" failed. No retries permitted until 2026-10-30 21:53:02.298137414 +0000 UTC m=+2.802809360 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8806f1e1-0057-4738-8fbc-7e503e1f91ca-metrics-certs") pod "network-metrics-daemon-wpdr9" (UID: "8806f1e1-0057-4738-8fbc-7e503e1f91ca") : secret "metrics-daemon-secret" not found
Oct 30 21:53:01.398679 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:01.398644    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/db073c3b-7cee-4e22-8c11-209bbeb21a96-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-bkhmz\" (UID: \"db073c3b-7cee-4e22-8c11-209bbeb21a96\") " pod="openshift-insights/insights-runtime-extractor-bkhmz"
Oct 30 21:53:01.398812 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:01.398690    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4a03a836-506f-404e-9c55-c26d779cd4ee-metrics-tls\") pod \"dns-default-8qxwh\" (UID: \"4a03a836-506f-404e-9c55-c26d779cd4ee\") " pod="openshift-dns/dns-default-8qxwh"
Oct 30 21:53:01.398812 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:01.398711    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/db073c3b-7cee-4e22-8c11-209bbeb21a96-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-bkhmz\" (UID: \"db073c3b-7cee-4e22-8c11-209bbeb21a96\") " pod="openshift-insights/insights-runtime-extractor-bkhmz"
Oct 30 21:53:01.398812 ip-10-0-131-202 kubenswrapper[4412]: E1030 21:53:01.398768    4412 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Oct 30 21:53:01.398812 ip-10-0-131-202 kubenswrapper[4412]: E1030 21:53:01.398789    4412 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Oct 30 21:53:01.398812 ip-10-0-131-202 kubenswrapper[4412]: E1030 21:53:01.398795    4412 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Oct 30 21:53:01.398968 ip-10-0-131-202 kubenswrapper[4412]: E1030 21:53:01.398829    4412 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/db073c3b-7cee-4e22-8c11-209bbeb21a96-kube-rbac-proxy-cm podName:db073c3b-7cee-4e22-8c11-209bbeb21a96 nodeName:}" failed. No retries permitted until 2026-10-30 21:53:02.398814466 +0000 UTC m=+2.903486401 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/db073c3b-7cee-4e22-8c11-209bbeb21a96-kube-rbac-proxy-cm") pod "insights-runtime-extractor-bkhmz" (UID: "db073c3b-7cee-4e22-8c11-209bbeb21a96") : configmap "kube-rbac-proxy" not found
Oct 30 21:53:01.398968 ip-10-0-131-202 kubenswrapper[4412]: E1030 21:53:01.398845    4412 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/db073c3b-7cee-4e22-8c11-209bbeb21a96-insights-runtime-extractor-tls podName:db073c3b-7cee-4e22-8c11-209bbeb21a96 nodeName:}" failed. No retries permitted until 2026-10-30 21:53:02.398839034 +0000 UTC m=+2.903510963 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/db073c3b-7cee-4e22-8c11-209bbeb21a96-insights-runtime-extractor-tls") pod "insights-runtime-extractor-bkhmz" (UID: "db073c3b-7cee-4e22-8c11-209bbeb21a96") : secret "insights-runtime-extractor-tls" not found
Oct 30 21:53:01.398968 ip-10-0-131-202 kubenswrapper[4412]: E1030 21:53:01.398855    4412 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4a03a836-506f-404e-9c55-c26d779cd4ee-metrics-tls podName:4a03a836-506f-404e-9c55-c26d779cd4ee nodeName:}" failed. No retries permitted until 2026-10-30 21:53:02.398849883 +0000 UTC m=+2.903521813 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4a03a836-506f-404e-9c55-c26d779cd4ee-metrics-tls") pod "dns-default-8qxwh" (UID: "4a03a836-506f-404e-9c55-c26d779cd4ee") : secret "dns-default-metrics-tls" not found
Oct 30 21:53:01.915825 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:01.915799    4412 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-lhj9s"
Oct 30 21:53:02.306963 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:02.306932    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8806f1e1-0057-4738-8fbc-7e503e1f91ca-metrics-certs\") pod \"network-metrics-daemon-wpdr9\" (UID: \"8806f1e1-0057-4738-8fbc-7e503e1f91ca\") " pod="openshift-multus/network-metrics-daemon-wpdr9"
Oct 30 21:53:02.307184 ip-10-0-131-202 kubenswrapper[4412]: E1030 21:53:02.307094    4412 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Oct 30 21:53:02.307254 ip-10-0-131-202 kubenswrapper[4412]: E1030 21:53:02.307196    4412 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8806f1e1-0057-4738-8fbc-7e503e1f91ca-metrics-certs podName:8806f1e1-0057-4738-8fbc-7e503e1f91ca nodeName:}" failed. No retries permitted until 2026-10-30 21:53:04.307171076 +0000 UTC m=+4.811843006 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8806f1e1-0057-4738-8fbc-7e503e1f91ca-metrics-certs") pod "network-metrics-daemon-wpdr9" (UID: "8806f1e1-0057-4738-8fbc-7e503e1f91ca") : secret "metrics-daemon-secret" not found
Oct 30 21:53:02.407770 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:02.407732    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/db073c3b-7cee-4e22-8c11-209bbeb21a96-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-bkhmz\" (UID: \"db073c3b-7cee-4e22-8c11-209bbeb21a96\") " pod="openshift-insights/insights-runtime-extractor-bkhmz"
Oct 30 21:53:02.407942 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:02.407782    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4a03a836-506f-404e-9c55-c26d779cd4ee-metrics-tls\") pod \"dns-default-8qxwh\" (UID: \"4a03a836-506f-404e-9c55-c26d779cd4ee\") " pod="openshift-dns/dns-default-8qxwh"
Oct 30 21:53:02.407942 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:02.407804    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/db073c3b-7cee-4e22-8c11-209bbeb21a96-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-bkhmz\" (UID: \"db073c3b-7cee-4e22-8c11-209bbeb21a96\") " pod="openshift-insights/insights-runtime-extractor-bkhmz"
Oct 30 21:53:02.407942 ip-10-0-131-202 kubenswrapper[4412]: E1030 21:53:02.407862    4412 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Oct 30 21:53:02.407942 ip-10-0-131-202 kubenswrapper[4412]: E1030 21:53:02.407892    4412 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Oct 30 21:53:02.407942 ip-10-0-131-202 kubenswrapper[4412]: E1030 21:53:02.407921    4412 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Oct 30 21:53:02.408106 ip-10-0-131-202 kubenswrapper[4412]: E1030 21:53:02.407942    4412 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/db073c3b-7cee-4e22-8c11-209bbeb21a96-kube-rbac-proxy-cm podName:db073c3b-7cee-4e22-8c11-209bbeb21a96 nodeName:}" failed. No retries permitted until 2026-10-30 21:53:04.40792559 +0000 UTC m=+4.912597540 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/db073c3b-7cee-4e22-8c11-209bbeb21a96-kube-rbac-proxy-cm") pod "insights-runtime-extractor-bkhmz" (UID: "db073c3b-7cee-4e22-8c11-209bbeb21a96") : configmap "kube-rbac-proxy" not found
Oct 30 21:53:02.408106 ip-10-0-131-202 kubenswrapper[4412]: E1030 21:53:02.407971    4412 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4a03a836-506f-404e-9c55-c26d779cd4ee-metrics-tls podName:4a03a836-506f-404e-9c55-c26d779cd4ee nodeName:}" failed. No retries permitted until 2026-10-30 21:53:04.407963482 +0000 UTC m=+4.912635411 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4a03a836-506f-404e-9c55-c26d779cd4ee-metrics-tls") pod "dns-default-8qxwh" (UID: "4a03a836-506f-404e-9c55-c26d779cd4ee") : secret "dns-default-metrics-tls" not found
Oct 30 21:53:02.408106 ip-10-0-131-202 kubenswrapper[4412]: E1030 21:53:02.407983    4412 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/db073c3b-7cee-4e22-8c11-209bbeb21a96-insights-runtime-extractor-tls podName:db073c3b-7cee-4e22-8c11-209bbeb21a96 nodeName:}" failed. No retries permitted until 2026-10-30 21:53:04.407975648 +0000 UTC m=+4.912647578 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/db073c3b-7cee-4e22-8c11-209bbeb21a96-insights-runtime-extractor-tls") pod "insights-runtime-extractor-bkhmz" (UID: "db073c3b-7cee-4e22-8c11-209bbeb21a96") : secret "insights-runtime-extractor-tls" not found
Oct 30 21:53:03.664784 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:03.664709    4412 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-zbfpb" event={"ID":"7e6b0df0-07eb-4301-adaa-1a6514639d9c","Type":"ContainerStarted","Data":"1e5afdb395c783ce025ae8bb09aabed61fec33aa568183e8f2b8cf81c6240c59"}
Oct 30 21:53:04.320850 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:04.320811    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8806f1e1-0057-4738-8fbc-7e503e1f91ca-metrics-certs\") pod \"network-metrics-daemon-wpdr9\" (UID: \"8806f1e1-0057-4738-8fbc-7e503e1f91ca\") " pod="openshift-multus/network-metrics-daemon-wpdr9"
Oct 30 21:53:04.321085 ip-10-0-131-202 kubenswrapper[4412]: E1030 21:53:04.320959    4412 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Oct 30 21:53:04.321085 ip-10-0-131-202 kubenswrapper[4412]: E1030 21:53:04.321024    4412 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8806f1e1-0057-4738-8fbc-7e503e1f91ca-metrics-certs podName:8806f1e1-0057-4738-8fbc-7e503e1f91ca nodeName:}" failed. No retries permitted until 2026-10-30 21:53:08.321010829 +0000 UTC m=+8.825682759 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8806f1e1-0057-4738-8fbc-7e503e1f91ca-metrics-certs") pod "network-metrics-daemon-wpdr9" (UID: "8806f1e1-0057-4738-8fbc-7e503e1f91ca") : secret "metrics-daemon-secret" not found
Oct 30 21:53:04.421809 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:04.421770    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4a03a836-506f-404e-9c55-c26d779cd4ee-metrics-tls\") pod \"dns-default-8qxwh\" (UID: \"4a03a836-506f-404e-9c55-c26d779cd4ee\") " pod="openshift-dns/dns-default-8qxwh"
Oct 30 21:53:04.421990 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:04.421818    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/db073c3b-7cee-4e22-8c11-209bbeb21a96-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-bkhmz\" (UID: \"db073c3b-7cee-4e22-8c11-209bbeb21a96\") " pod="openshift-insights/insights-runtime-extractor-bkhmz"
Oct 30 21:53:04.421990 ip-10-0-131-202 kubenswrapper[4412]: E1030 21:53:04.421931    4412 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Oct 30 21:53:04.421990 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:04.421969    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/db073c3b-7cee-4e22-8c11-209bbeb21a96-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-bkhmz\" (UID: \"db073c3b-7cee-4e22-8c11-209bbeb21a96\") " pod="openshift-insights/insights-runtime-extractor-bkhmz"
Oct 30 21:53:04.421990 ip-10-0-131-202 kubenswrapper[4412]: E1030 21:53:04.421990    4412 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/db073c3b-7cee-4e22-8c11-209bbeb21a96-insights-runtime-extractor-tls podName:db073c3b-7cee-4e22-8c11-209bbeb21a96 nodeName:}" failed. No retries permitted until 2026-10-30 21:53:08.421973642 +0000 UTC m=+8.926645578 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/db073c3b-7cee-4e22-8c11-209bbeb21a96-insights-runtime-extractor-tls") pod "insights-runtime-extractor-bkhmz" (UID: "db073c3b-7cee-4e22-8c11-209bbeb21a96") : secret "insights-runtime-extractor-tls" not found
Oct 30 21:53:04.422204 ip-10-0-131-202 kubenswrapper[4412]: E1030 21:53:04.421931    4412 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Oct 30 21:53:04.422204 ip-10-0-131-202 kubenswrapper[4412]: E1030 21:53:04.422060    4412 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4a03a836-506f-404e-9c55-c26d779cd4ee-metrics-tls podName:4a03a836-506f-404e-9c55-c26d779cd4ee nodeName:}" failed. No retries permitted until 2026-10-30 21:53:08.42205034 +0000 UTC m=+8.926722292 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4a03a836-506f-404e-9c55-c26d779cd4ee-metrics-tls") pod "dns-default-8qxwh" (UID: "4a03a836-506f-404e-9c55-c26d779cd4ee") : secret "dns-default-metrics-tls" not found
Oct 30 21:53:04.422545 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:04.422527    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/db073c3b-7cee-4e22-8c11-209bbeb21a96-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-bkhmz\" (UID: \"db073c3b-7cee-4e22-8c11-209bbeb21a96\") " pod="openshift-insights/insights-runtime-extractor-bkhmz"
Oct 30 21:53:05.694258 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:05.694227    4412 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-lhj9s"
Oct 30 21:53:05.694630 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:05.694487    4412 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Oct 30 21:53:08.190579 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:08.190552    4412 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-7v7fb_dffd7a70-b7f3-4e1b-b3ab-74c5a574cab2/dns-node-resolver/0.log"
Oct 30 21:53:08.348293 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:08.348260    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8806f1e1-0057-4738-8fbc-7e503e1f91ca-metrics-certs\") pod \"network-metrics-daemon-wpdr9\" (UID: \"8806f1e1-0057-4738-8fbc-7e503e1f91ca\") " pod="openshift-multus/network-metrics-daemon-wpdr9"
Oct 30 21:53:08.348484 ip-10-0-131-202 kubenswrapper[4412]: E1030 21:53:08.348374    4412 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Oct 30 21:53:08.348484 ip-10-0-131-202 kubenswrapper[4412]: E1030 21:53:08.348428    4412 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8806f1e1-0057-4738-8fbc-7e503e1f91ca-metrics-certs podName:8806f1e1-0057-4738-8fbc-7e503e1f91ca nodeName:}" failed. No retries permitted until 2026-10-30 21:53:16.34841604 +0000 UTC m=+16.853087970 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8806f1e1-0057-4738-8fbc-7e503e1f91ca-metrics-certs") pod "network-metrics-daemon-wpdr9" (UID: "8806f1e1-0057-4738-8fbc-7e503e1f91ca") : secret "metrics-daemon-secret" not found
Oct 30 21:53:08.448923 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:08.448841    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4a03a836-506f-404e-9c55-c26d779cd4ee-metrics-tls\") pod \"dns-default-8qxwh\" (UID: \"4a03a836-506f-404e-9c55-c26d779cd4ee\") " pod="openshift-dns/dns-default-8qxwh"
Oct 30 21:53:08.448923 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:08.448878    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/db073c3b-7cee-4e22-8c11-209bbeb21a96-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-bkhmz\" (UID: \"db073c3b-7cee-4e22-8c11-209bbeb21a96\") " pod="openshift-insights/insights-runtime-extractor-bkhmz"
Oct 30 21:53:08.449159 ip-10-0-131-202 kubenswrapper[4412]: E1030 21:53:08.448980    4412 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Oct 30 21:53:08.449159 ip-10-0-131-202 kubenswrapper[4412]: E1030 21:53:08.449005    4412 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Oct 30 21:53:08.449159 ip-10-0-131-202 kubenswrapper[4412]: E1030 21:53:08.449027    4412 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/db073c3b-7cee-4e22-8c11-209bbeb21a96-insights-runtime-extractor-tls podName:db073c3b-7cee-4e22-8c11-209bbeb21a96 nodeName:}" failed. No retries permitted until 2026-10-30 21:53:16.4490156 +0000 UTC m=+16.953687543 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/db073c3b-7cee-4e22-8c11-209bbeb21a96-insights-runtime-extractor-tls") pod "insights-runtime-extractor-bkhmz" (UID: "db073c3b-7cee-4e22-8c11-209bbeb21a96") : secret "insights-runtime-extractor-tls" not found
Oct 30 21:53:08.449159 ip-10-0-131-202 kubenswrapper[4412]: E1030 21:53:08.449077    4412 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4a03a836-506f-404e-9c55-c26d779cd4ee-metrics-tls podName:4a03a836-506f-404e-9c55-c26d779cd4ee nodeName:}" failed. No retries permitted until 2026-10-30 21:53:16.449060305 +0000 UTC m=+16.953732251 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4a03a836-506f-404e-9c55-c26d779cd4ee-metrics-tls") pod "dns-default-8qxwh" (UID: "4a03a836-506f-404e-9c55-c26d779cd4ee") : secret "dns-default-metrics-tls" not found
Oct 30 21:53:09.392166 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:09.392136    4412 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-f9t6t_164bafd8-ff17-46a2-afc0-dd01fa1aaeb6/node-ca/0.log"
Oct 30 21:53:10.190869 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:10.190836    4412 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-h2pjp_2345ff5b-db97-4855-8adb-59b9a780615e/serve-healthcheck-canary/0.log"
Oct 30 21:53:10.797856 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:10.797826    4412 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-66df5d496b-zbfpb_7e6b0df0-07eb-4301-adaa-1a6514639d9c/migrator/0.log"
Oct 30 21:53:10.828625 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:10.828594    4412 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-slfvl"
Oct 30 21:53:16.401010 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:16.400968    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8806f1e1-0057-4738-8fbc-7e503e1f91ca-metrics-certs\") pod \"network-metrics-daemon-wpdr9\" (UID: \"8806f1e1-0057-4738-8fbc-7e503e1f91ca\") " pod="openshift-multus/network-metrics-daemon-wpdr9"
Oct 30 21:53:16.401524 ip-10-0-131-202 kubenswrapper[4412]: E1030 21:53:16.401141    4412 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Oct 30 21:53:16.401524 ip-10-0-131-202 kubenswrapper[4412]: E1030 21:53:16.401208    4412 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8806f1e1-0057-4738-8fbc-7e503e1f91ca-metrics-certs podName:8806f1e1-0057-4738-8fbc-7e503e1f91ca nodeName:}" failed. No retries permitted until 2026-10-30 21:53:32.401192445 +0000 UTC m=+32.905864374 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8806f1e1-0057-4738-8fbc-7e503e1f91ca-metrics-certs") pod "network-metrics-daemon-wpdr9" (UID: "8806f1e1-0057-4738-8fbc-7e503e1f91ca") : secret "metrics-daemon-secret" not found
Oct 30 21:53:16.502217 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:16.502187    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4a03a836-506f-404e-9c55-c26d779cd4ee-metrics-tls\") pod \"dns-default-8qxwh\" (UID: \"4a03a836-506f-404e-9c55-c26d779cd4ee\") " pod="openshift-dns/dns-default-8qxwh"
Oct 30 21:53:16.502329 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:16.502227    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/db073c3b-7cee-4e22-8c11-209bbeb21a96-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-bkhmz\" (UID: \"db073c3b-7cee-4e22-8c11-209bbeb21a96\") " pod="openshift-insights/insights-runtime-extractor-bkhmz"
Oct 30 21:53:16.502613 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:16.502593    4412 swap_util.go:78] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Oct 30 21:53:16.504767 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:16.504743    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/db073c3b-7cee-4e22-8c11-209bbeb21a96-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-bkhmz\" (UID: \"db073c3b-7cee-4e22-8c11-209bbeb21a96\") " pod="openshift-insights/insights-runtime-extractor-bkhmz"
Oct 30 21:53:16.504873 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:16.504861    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4a03a836-506f-404e-9c55-c26d779cd4ee-metrics-tls\") pod \"dns-default-8qxwh\" (UID: \"4a03a836-506f-404e-9c55-c26d779cd4ee\") " pod="openshift-dns/dns-default-8qxwh"
Oct 30 21:53:16.583937 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:16.583895    4412 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-8qxwh"
Oct 30 21:53:16.584075 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:16.583900    4412 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-bkhmz"
Oct 30 21:53:17.061289 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:17.061258    4412 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-8qxwh"]
Oct 30 21:53:17.064714 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:53:17.064682    4412 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4a03a836_506f_404e_9c55_c26d779cd4ee.slice/crio-32ebf68946db163e87d8968bd8a002c306ec3a239aa37d78b46e5e322d74e703 WatchSource:0}: Error finding container 32ebf68946db163e87d8968bd8a002c306ec3a239aa37d78b46e5e322d74e703: Status 404 returned error can't find the container with id 32ebf68946db163e87d8968bd8a002c306ec3a239aa37d78b46e5e322d74e703
Oct 30 21:53:17.111703 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:17.111679    4412 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-bkhmz"]
Oct 30 21:53:17.113793 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:53:17.113770    4412 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddb073c3b_7cee_4e22_8c11_209bbeb21a96.slice/crio-a0c8e1aca5990227af7298c26e32e5b11ad736444b389695f9b57a3231ab0a92 WatchSource:0}: Error finding container a0c8e1aca5990227af7298c26e32e5b11ad736444b389695f9b57a3231ab0a92: Status 404 returned error can't find the container with id a0c8e1aca5990227af7298c26e32e5b11ad736444b389695f9b57a3231ab0a92
Oct 30 21:53:17.698049 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:17.698010    4412 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-8qxwh" event={"ID":"4a03a836-506f-404e-9c55-c26d779cd4ee","Type":"ContainerStarted","Data":"32ebf68946db163e87d8968bd8a002c306ec3a239aa37d78b46e5e322d74e703"}
Oct 30 21:53:17.699554 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:17.699525    4412 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-bkhmz" event={"ID":"db073c3b-7cee-4e22-8c11-209bbeb21a96","Type":"ContainerStarted","Data":"bfabda01ffacc920b7b1cfc3ffaa5950949c0024ed782e8e9bb3c62b9dc66b6f"}
Oct 30 21:53:17.699674 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:17.699578    4412 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-bkhmz" event={"ID":"db073c3b-7cee-4e22-8c11-209bbeb21a96","Type":"ContainerStarted","Data":"a0c8e1aca5990227af7298c26e32e5b11ad736444b389695f9b57a3231ab0a92"}
Oct 30 21:53:18.704553 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:18.704475    4412 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-bkhmz" event={"ID":"db073c3b-7cee-4e22-8c11-209bbeb21a96","Type":"ContainerStarted","Data":"8dd7d5f48e38c0d0098f48000728f9dea157a566991c9fc8472ee5670d8926db"}
Oct 30 21:53:19.710175 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:19.710103    4412 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-8qxwh" event={"ID":"4a03a836-506f-404e-9c55-c26d779cd4ee","Type":"ContainerStarted","Data":"f0c964a8d7886159ec7ee610042a866b42b13c7b28b4fb9011bf4e55a988b2d3"}
Oct 30 21:53:19.710661 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:19.710637    4412 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-8qxwh"
Oct 30 21:53:19.710759 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:19.710701    4412 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-8qxwh" event={"ID":"4a03a836-506f-404e-9c55-c26d779cd4ee","Type":"ContainerStarted","Data":"094c4a10ab2f0ad6e651b280f639850bcb5ac60e0d118af7b69ea6716a7294f0"}
Oct 30 21:53:20.715745 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:20.715704    4412 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-bkhmz" event={"ID":"db073c3b-7cee-4e22-8c11-209bbeb21a96","Type":"ContainerStarted","Data":"1ebc63e79785dc1bbab41e851b5cb359214456b11f0d0d24eeb8e56d5fe9626f"}
Oct 30 21:53:23.382185 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:23.382153    4412 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-54cc95575d-b877n"]
Oct 30 21:53:23.384048 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:23.384026    4412 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-54cc95575d-b877n"
Oct 30 21:53:23.450025 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:23.449998    4412 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/81d63130-0f57-4fee-aace-450bd71391f6-ca-trust-extracted\") pod \"image-registry-54cc95575d-b877n\" (UID: \"81d63130-0f57-4fee-aace-450bd71391f6\") " pod="openshift-image-registry/image-registry-54cc95575d-b877n"
Oct 30 21:53:23.450025 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:23.450029    4412 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/81d63130-0f57-4fee-aace-450bd71391f6-registry-certificates\") pod \"image-registry-54cc95575d-b877n\" (UID: \"81d63130-0f57-4fee-aace-450bd71391f6\") " pod="openshift-image-registry/image-registry-54cc95575d-b877n"
Oct 30 21:53:23.450248 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:23.450050    4412 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/81d63130-0f57-4fee-aace-450bd71391f6-registry-tls\") pod \"image-registry-54cc95575d-b877n\" (UID: \"81d63130-0f57-4fee-aace-450bd71391f6\") " pod="openshift-image-registry/image-registry-54cc95575d-b877n"
Oct 30 21:53:23.450248 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:23.450073    4412 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/81d63130-0f57-4fee-aace-450bd71391f6-trusted-ca\") pod \"image-registry-54cc95575d-b877n\" (UID: \"81d63130-0f57-4fee-aace-450bd71391f6\") " pod="openshift-image-registry/image-registry-54cc95575d-b877n"
Oct 30 21:53:23.450248 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:23.450178    4412 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w6g89\" (UniqueName: \"kubernetes.io/projected/81d63130-0f57-4fee-aace-450bd71391f6-kube-api-access-w6g89\") pod \"image-registry-54cc95575d-b877n\" (UID: \"81d63130-0f57-4fee-aace-450bd71391f6\") " pod="openshift-image-registry/image-registry-54cc95575d-b877n"
Oct 30 21:53:23.450248 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:23.450209    4412 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/81d63130-0f57-4fee-aace-450bd71391f6-image-registry-private-configuration\") pod \"image-registry-54cc95575d-b877n\" (UID: \"81d63130-0f57-4fee-aace-450bd71391f6\") " pod="openshift-image-registry/image-registry-54cc95575d-b877n"
Oct 30 21:53:23.450248 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:23.450233    4412 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/81d63130-0f57-4fee-aace-450bd71391f6-installation-pull-secrets\") pod \"image-registry-54cc95575d-b877n\" (UID: \"81d63130-0f57-4fee-aace-450bd71391f6\") " pod="openshift-image-registry/image-registry-54cc95575d-b877n"
Oct 30 21:53:23.450422 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:23.450282    4412 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/81d63130-0f57-4fee-aace-450bd71391f6-bound-sa-token\") pod \"image-registry-54cc95575d-b877n\" (UID: \"81d63130-0f57-4fee-aace-450bd71391f6\") " pod="openshift-image-registry/image-registry-54cc95575d-b877n"
Oct 30 21:53:23.550961 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:23.550932    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/81d63130-0f57-4fee-aace-450bd71391f6-image-registry-private-configuration\") pod \"image-registry-54cc95575d-b877n\" (UID: \"81d63130-0f57-4fee-aace-450bd71391f6\") " pod="openshift-image-registry/image-registry-54cc95575d-b877n"
Oct 30 21:53:23.551182 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:23.550976    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/81d63130-0f57-4fee-aace-450bd71391f6-installation-pull-secrets\") pod \"image-registry-54cc95575d-b877n\" (UID: \"81d63130-0f57-4fee-aace-450bd71391f6\") " pod="openshift-image-registry/image-registry-54cc95575d-b877n"
Oct 30 21:53:23.551182 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:23.551000    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/81d63130-0f57-4fee-aace-450bd71391f6-bound-sa-token\") pod \"image-registry-54cc95575d-b877n\" (UID: \"81d63130-0f57-4fee-aace-450bd71391f6\") " pod="openshift-image-registry/image-registry-54cc95575d-b877n"
Oct 30 21:53:23.551182 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:23.551141    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/81d63130-0f57-4fee-aace-450bd71391f6-ca-trust-extracted\") pod \"image-registry-54cc95575d-b877n\" (UID: \"81d63130-0f57-4fee-aace-450bd71391f6\") " pod="openshift-image-registry/image-registry-54cc95575d-b877n"
Oct 30 21:53:23.551182 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:23.551168    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/81d63130-0f57-4fee-aace-450bd71391f6-registry-certificates\") pod \"image-registry-54cc95575d-b877n\" (UID: \"81d63130-0f57-4fee-aace-450bd71391f6\") " pod="openshift-image-registry/image-registry-54cc95575d-b877n"
Oct 30 21:53:23.551410 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:23.551190    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/81d63130-0f57-4fee-aace-450bd71391f6-registry-tls\") pod \"image-registry-54cc95575d-b877n\" (UID: \"81d63130-0f57-4fee-aace-450bd71391f6\") " pod="openshift-image-registry/image-registry-54cc95575d-b877n"
Oct 30 21:53:23.551410 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:23.551217    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/81d63130-0f57-4fee-aace-450bd71391f6-trusted-ca\") pod \"image-registry-54cc95575d-b877n\" (UID: \"81d63130-0f57-4fee-aace-450bd71391f6\") " pod="openshift-image-registry/image-registry-54cc95575d-b877n"
Oct 30 21:53:23.551410 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:23.551235    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w6g89\" (UniqueName: \"kubernetes.io/projected/81d63130-0f57-4fee-aace-450bd71391f6-kube-api-access-w6g89\") pod \"image-registry-54cc95575d-b877n\" (UID: \"81d63130-0f57-4fee-aace-450bd71391f6\") " pod="openshift-image-registry/image-registry-54cc95575d-b877n"
Oct 30 21:53:23.551578 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:23.551561    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/81d63130-0f57-4fee-aace-450bd71391f6-ca-trust-extracted\") pod \"image-registry-54cc95575d-b877n\" (UID: \"81d63130-0f57-4fee-aace-450bd71391f6\") " pod="openshift-image-registry/image-registry-54cc95575d-b877n"
Oct 30 21:53:23.551941 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:23.551922    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/81d63130-0f57-4fee-aace-450bd71391f6-registry-certificates\") pod \"image-registry-54cc95575d-b877n\" (UID: \"81d63130-0f57-4fee-aace-450bd71391f6\") " pod="openshift-image-registry/image-registry-54cc95575d-b877n"
Oct 30 21:53:23.598951 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:23.598922    4412 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-fsvt2\""
Oct 30 21:53:23.599086 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:23.599075    4412 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Oct 30 21:53:23.599200 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:23.599185    4412 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Oct 30 21:53:23.599385 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:23.599370    4412 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Oct 30 21:53:23.599955 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:23.599928    4412 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Oct 30 21:53:23.601380 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:23.601314    4412 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-54cc95575d-b877n"]
Oct 30 21:53:23.602547 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:23.602498    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/81d63130-0f57-4fee-aace-450bd71391f6-trusted-ca\") pod \"image-registry-54cc95575d-b877n\" (UID: \"81d63130-0f57-4fee-aace-450bd71391f6\") " pod="openshift-image-registry/image-registry-54cc95575d-b877n"
Oct 30 21:53:23.604299 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:23.604279    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/81d63130-0f57-4fee-aace-450bd71391f6-image-registry-private-configuration\") pod \"image-registry-54cc95575d-b877n\" (UID: \"81d63130-0f57-4fee-aace-450bd71391f6\") " pod="openshift-image-registry/image-registry-54cc95575d-b877n"
Oct 30 21:53:23.604405 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:23.604385    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/81d63130-0f57-4fee-aace-450bd71391f6-installation-pull-secrets\") pod \"image-registry-54cc95575d-b877n\" (UID: \"81d63130-0f57-4fee-aace-450bd71391f6\") " pod="openshift-image-registry/image-registry-54cc95575d-b877n"
Oct 30 21:53:23.604447 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:23.604394    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/81d63130-0f57-4fee-aace-450bd71391f6-registry-tls\") pod \"image-registry-54cc95575d-b877n\" (UID: \"81d63130-0f57-4fee-aace-450bd71391f6\") " pod="openshift-image-registry/image-registry-54cc95575d-b877n"
Oct 30 21:53:23.609090 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:23.609069    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/81d63130-0f57-4fee-aace-450bd71391f6-bound-sa-token\") pod \"image-registry-54cc95575d-b877n\" (UID: \"81d63130-0f57-4fee-aace-450bd71391f6\") " pod="openshift-image-registry/image-registry-54cc95575d-b877n"
Oct 30 21:53:23.609570 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:23.609547    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w6g89\" (UniqueName: \"kubernetes.io/projected/81d63130-0f57-4fee-aace-450bd71391f6-kube-api-access-w6g89\") pod \"image-registry-54cc95575d-b877n\" (UID: \"81d63130-0f57-4fee-aace-450bd71391f6\") " pod="openshift-image-registry/image-registry-54cc95575d-b877n"
Oct 30 21:53:23.693977 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:23.693893    4412 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-54cc95575d-b877n"
Oct 30 21:53:24.166845 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:24.166818    4412 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-54cc95575d-b877n"]
Oct 30 21:53:24.169701 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:53:24.169669    4412 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod81d63130_0f57_4fee_aace_450bd71391f6.slice/crio-9df69331192c8ab0e543bb65a0d4893648878aa8b0ea929d48c477862fffb00d WatchSource:0}: Error finding container 9df69331192c8ab0e543bb65a0d4893648878aa8b0ea929d48c477862fffb00d: Status 404 returned error can't find the container with id 9df69331192c8ab0e543bb65a0d4893648878aa8b0ea929d48c477862fffb00d
Oct 30 21:53:24.728540 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:24.728506    4412 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-54cc95575d-b877n" event={"ID":"81d63130-0f57-4fee-aace-450bd71391f6","Type":"ContainerStarted","Data":"9532ee7806fd3182d341ecc07c2c6b13edb2117178b585ecfe930fa60fce8473"}
Oct 30 21:53:24.728962 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:24.728553    4412 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-54cc95575d-b877n" event={"ID":"81d63130-0f57-4fee-aace-450bd71391f6","Type":"ContainerStarted","Data":"9df69331192c8ab0e543bb65a0d4893648878aa8b0ea929d48c477862fffb00d"}
Oct 30 21:53:24.728962 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:24.728640    4412 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-54cc95575d-b877n"
Oct 30 21:53:24.850316 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:24.850256    4412 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-image-registry/image-registry-54cc95575d-b877n" podStartSLOduration=1.8502427080000001 podStartE2EDuration="1.850242708s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:53:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:53:24.849817178 +0000 UTC m=+25.354489152" watchObservedRunningTime="2026-10-30 21:53:24.850242708 +0000 UTC m=+25.354914660"
Oct 30 21:53:27.645784 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:27.645757    4412 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-x7l5l"
Oct 30 21:53:28.697106 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:28.697072    4412 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-9cd89b74c-plp26"]
Oct 30 21:53:28.742439 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:28.742412    4412 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-9cd89b74c-plp26"
Oct 30 21:53:28.758765 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:28.758735    4412 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-9cd89b74c-plp26"]
Oct 30 21:53:28.786760 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:28.786734    4412 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0886e99f-8cba-48a7-9dae-e13d46230fac-metrics-client-ca\") pod \"prometheus-operator-9cd89b74c-plp26\" (UID: \"0886e99f-8cba-48a7-9dae-e13d46230fac\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-plp26"
Oct 30 21:53:28.786890 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:28.786796    4412 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/0886e99f-8cba-48a7-9dae-e13d46230fac-prometheus-operator-tls\") pod \"prometheus-operator-9cd89b74c-plp26\" (UID: \"0886e99f-8cba-48a7-9dae-e13d46230fac\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-plp26"
Oct 30 21:53:28.786934 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:28.786895    4412 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0886e99f-8cba-48a7-9dae-e13d46230fac-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-9cd89b74c-plp26\" (UID: \"0886e99f-8cba-48a7-9dae-e13d46230fac\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-plp26"
Oct 30 21:53:28.786934 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:28.786916    4412 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lzll6\" (UniqueName: \"kubernetes.io/projected/0886e99f-8cba-48a7-9dae-e13d46230fac-kube-api-access-lzll6\") pod \"prometheus-operator-9cd89b74c-plp26\" (UID: \"0886e99f-8cba-48a7-9dae-e13d46230fac\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-plp26"
Oct 30 21:53:28.808487 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:28.808464    4412 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Oct 30 21:53:28.808623 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:28.808495    4412 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\""
Oct 30 21:53:28.858825 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:28.858794    4412 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\""
Oct 30 21:53:28.858943 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:28.858826    4412 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Oct 30 21:53:28.858943 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:28.858859    4412 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-dskxg\""
Oct 30 21:53:28.858943 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:28.858897    4412 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Oct 30 21:53:28.888210 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:28.888188    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/0886e99f-8cba-48a7-9dae-e13d46230fac-prometheus-operator-tls\") pod \"prometheus-operator-9cd89b74c-plp26\" (UID: \"0886e99f-8cba-48a7-9dae-e13d46230fac\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-plp26"
Oct 30 21:53:28.888346 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:28.888218    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0886e99f-8cba-48a7-9dae-e13d46230fac-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-9cd89b74c-plp26\" (UID: \"0886e99f-8cba-48a7-9dae-e13d46230fac\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-plp26"
Oct 30 21:53:28.888346 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:28.888237    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lzll6\" (UniqueName: \"kubernetes.io/projected/0886e99f-8cba-48a7-9dae-e13d46230fac-kube-api-access-lzll6\") pod \"prometheus-operator-9cd89b74c-plp26\" (UID: \"0886e99f-8cba-48a7-9dae-e13d46230fac\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-plp26"
Oct 30 21:53:28.888346 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:28.888266    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0886e99f-8cba-48a7-9dae-e13d46230fac-metrics-client-ca\") pod \"prometheus-operator-9cd89b74c-plp26\" (UID: \"0886e99f-8cba-48a7-9dae-e13d46230fac\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-plp26"
Oct 30 21:53:28.888841 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:28.888822    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0886e99f-8cba-48a7-9dae-e13d46230fac-metrics-client-ca\") pod \"prometheus-operator-9cd89b74c-plp26\" (UID: \"0886e99f-8cba-48a7-9dae-e13d46230fac\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-plp26"
Oct 30 21:53:28.890857 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:28.890836    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/0886e99f-8cba-48a7-9dae-e13d46230fac-prometheus-operator-tls\") pod \"prometheus-operator-9cd89b74c-plp26\" (UID: \"0886e99f-8cba-48a7-9dae-e13d46230fac\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-plp26"
Oct 30 21:53:28.890857 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:28.890849    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0886e99f-8cba-48a7-9dae-e13d46230fac-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-9cd89b74c-plp26\" (UID: \"0886e99f-8cba-48a7-9dae-e13d46230fac\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-plp26"
Oct 30 21:53:28.946070 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:28.946042    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lzll6\" (UniqueName: \"kubernetes.io/projected/0886e99f-8cba-48a7-9dae-e13d46230fac-kube-api-access-lzll6\") pod \"prometheus-operator-9cd89b74c-plp26\" (UID: \"0886e99f-8cba-48a7-9dae-e13d46230fac\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-plp26"
Oct 30 21:53:29.081401 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:29.081366    4412 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-9cd89b74c-plp26"
Oct 30 21:53:29.481370 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:29.481343    4412 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-9cd89b74c-plp26"]
Oct 30 21:53:29.485950 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:53:29.485923    4412 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0886e99f_8cba_48a7_9dae_e13d46230fac.slice/crio-fbc8ba3f93ca0d890e7bdf8811d0928e92420adfe67090133069381aa7856bce WatchSource:0}: Error finding container fbc8ba3f93ca0d890e7bdf8811d0928e92420adfe67090133069381aa7856bce: Status 404 returned error can't find the container with id fbc8ba3f93ca0d890e7bdf8811d0928e92420adfe67090133069381aa7856bce
Oct 30 21:53:29.717679 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:29.717653    4412 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-8qxwh"
Oct 30 21:53:29.743867 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:29.743788    4412 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-9cd89b74c-plp26" event={"ID":"0886e99f-8cba-48a7-9dae-e13d46230fac","Type":"ContainerStarted","Data":"fbc8ba3f93ca0d890e7bdf8811d0928e92420adfe67090133069381aa7856bce"}
Oct 30 21:53:29.992673 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:29.992645    4412 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-lhj9s"
Oct 30 21:53:30.831301 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:30.831275    4412 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-slfvl"
Oct 30 21:53:31.752737 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:31.752710    4412 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-9cd89b74c-plp26" event={"ID":"0886e99f-8cba-48a7-9dae-e13d46230fac","Type":"ContainerStarted","Data":"21fd0262e1862edb257393e39396089326856a9f863fcbff2a7328f427962079"}
Oct 30 21:53:31.752928 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:31.752765    4412 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-9cd89b74c-plp26" event={"ID":"0886e99f-8cba-48a7-9dae-e13d46230fac","Type":"ContainerStarted","Data":"c07e22128e4ed522d5abf0f22554f96dccf9b9f9be2f84fe76e5deadee96a006"}
Oct 30 21:53:31.900390 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:31.900294    4412 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-9cd89b74c-plp26" podStartSLOduration=1.7684199459999999 podStartE2EDuration="3.900281304s" totalImagesPullingTime="2.131861358s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:53:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:53:31.899959883 +0000 UTC m=+32.404631836" watchObservedRunningTime="2026-10-30 21:53:31.900281304 +0000 UTC m=+32.404953253"
Oct 30 21:53:32.416155 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:32.416106    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8806f1e1-0057-4738-8fbc-7e503e1f91ca-metrics-certs\") pod \"network-metrics-daemon-wpdr9\" (UID: \"8806f1e1-0057-4738-8fbc-7e503e1f91ca\") " pod="openshift-multus/network-metrics-daemon-wpdr9"
Oct 30 21:53:32.418482 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:32.418461    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8806f1e1-0057-4738-8fbc-7e503e1f91ca-metrics-certs\") pod \"network-metrics-daemon-wpdr9\" (UID: \"8806f1e1-0057-4738-8fbc-7e503e1f91ca\") " pod="openshift-multus/network-metrics-daemon-wpdr9"
Oct 30 21:53:32.476997 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:32.476968    4412 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wpdr9"
Oct 30 21:53:32.843022 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:32.842993    4412 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-wpdr9"]
Oct 30 21:53:32.845871 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:53:32.845839    4412 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8806f1e1_0057_4738_8fbc_7e503e1f91ca.slice/crio-c33f92e03fbcaa6d0d4bafdf12bab8d30849bdcc80d0736e7a3ae9b15ffcb5a7 WatchSource:0}: Error finding container c33f92e03fbcaa6d0d4bafdf12bab8d30849bdcc80d0736e7a3ae9b15ffcb5a7: Status 404 returned error can't find the container with id c33f92e03fbcaa6d0d4bafdf12bab8d30849bdcc80d0736e7a3ae9b15ffcb5a7
Oct 30 21:53:33.761629 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:33.761591    4412 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-wpdr9" event={"ID":"8806f1e1-0057-4738-8fbc-7e503e1f91ca","Type":"ContainerStarted","Data":"c33f92e03fbcaa6d0d4bafdf12bab8d30849bdcc80d0736e7a3ae9b15ffcb5a7"}
Oct 30 21:53:34.448154 ip-10-0-131-202 kubenswrapper[4412]: E1030 21:53:34.448101    4412 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bkhmz" containerName="extractor"
Oct 30 21:53:34.766745 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:34.766709    4412 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-wpdr9" event={"ID":"8806f1e1-0057-4738-8fbc-7e503e1f91ca","Type":"ContainerStarted","Data":"000330b2e6fb4c517246fcb3be89d83b2b2f5fde53340bb0235333f1fbaa4f52"}
Oct 30 21:53:34.767200 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:34.766783    4412 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-wpdr9" event={"ID":"8806f1e1-0057-4738-8fbc-7e503e1f91ca","Type":"ContainerStarted","Data":"d61341a6969584622f675e613da9d28d451b8aabacb3675d25f25c101f9b7f4d"}
Oct 30 21:53:35.151861 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:35.151784    4412 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-7bd7489957-5gm8c"]
Oct 30 21:53:35.153826 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:35.153809    4412 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-5gm8c"
Oct 30 21:53:35.236760 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:35.236732    4412 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/b9c973ab-4163-413a-8969-2947502f3592-openshift-state-metrics-tls\") pod \"openshift-state-metrics-7bd7489957-5gm8c\" (UID: \"b9c973ab-4163-413a-8969-2947502f3592\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-5gm8c"
Oct 30 21:53:35.236912 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:35.236774    4412 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zvr2l\" (UniqueName: \"kubernetes.io/projected/b9c973ab-4163-413a-8969-2947502f3592-kube-api-access-zvr2l\") pod \"openshift-state-metrics-7bd7489957-5gm8c\" (UID: \"b9c973ab-4163-413a-8969-2947502f3592\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-5gm8c"
Oct 30 21:53:35.236912 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:35.236840    4412 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b9c973ab-4163-413a-8969-2947502f3592-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-7bd7489957-5gm8c\" (UID: \"b9c973ab-4163-413a-8969-2947502f3592\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-5gm8c"
Oct 30 21:53:35.236912 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:35.236858    4412 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b9c973ab-4163-413a-8969-2947502f3592-metrics-client-ca\") pod \"openshift-state-metrics-7bd7489957-5gm8c\" (UID: \"b9c973ab-4163-413a-8969-2947502f3592\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-5gm8c"
Oct 30 21:53:35.257752 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:35.257729    4412 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-n9wgp\""
Oct 30 21:53:35.257875 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:35.257859    4412 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\""
Oct 30 21:53:35.257914 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:35.257905    4412 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\""
Oct 30 21:53:35.258665 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:35.258649    4412 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-7bd7489957-5gm8c"]
Oct 30 21:53:35.309564 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:35.309530    4412 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-ckb8k"]
Oct 30 21:53:35.311509 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:35.311488    4412 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-ckb8k"
Oct 30 21:53:35.337807 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:35.337783    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zvr2l\" (UniqueName: \"kubernetes.io/projected/b9c973ab-4163-413a-8969-2947502f3592-kube-api-access-zvr2l\") pod \"openshift-state-metrics-7bd7489957-5gm8c\" (UID: \"b9c973ab-4163-413a-8969-2947502f3592\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-5gm8c"
Oct 30 21:53:35.337931 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:35.337822    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b9c973ab-4163-413a-8969-2947502f3592-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-7bd7489957-5gm8c\" (UID: \"b9c973ab-4163-413a-8969-2947502f3592\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-5gm8c"
Oct 30 21:53:35.337931 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:35.337840    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b9c973ab-4163-413a-8969-2947502f3592-metrics-client-ca\") pod \"openshift-state-metrics-7bd7489957-5gm8c\" (UID: \"b9c973ab-4163-413a-8969-2947502f3592\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-5gm8c"
Oct 30 21:53:35.337931 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:35.337878    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/b9c973ab-4163-413a-8969-2947502f3592-openshift-state-metrics-tls\") pod \"openshift-state-metrics-7bd7489957-5gm8c\" (UID: \"b9c973ab-4163-413a-8969-2947502f3592\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-5gm8c"
Oct 30 21:53:35.338566 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:35.338549    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b9c973ab-4163-413a-8969-2947502f3592-metrics-client-ca\") pod \"openshift-state-metrics-7bd7489957-5gm8c\" (UID: \"b9c973ab-4163-413a-8969-2947502f3592\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-5gm8c"
Oct 30 21:53:35.340259 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:35.340239    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b9c973ab-4163-413a-8969-2947502f3592-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-7bd7489957-5gm8c\" (UID: \"b9c973ab-4163-413a-8969-2947502f3592\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-5gm8c"
Oct 30 21:53:35.340420 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:35.340400    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/b9c973ab-4163-413a-8969-2947502f3592-openshift-state-metrics-tls\") pod \"openshift-state-metrics-7bd7489957-5gm8c\" (UID: \"b9c973ab-4163-413a-8969-2947502f3592\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-5gm8c"
Oct 30 21:53:35.401706 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:35.401675    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zvr2l\" (UniqueName: \"kubernetes.io/projected/b9c973ab-4163-413a-8969-2947502f3592-kube-api-access-zvr2l\") pod \"openshift-state-metrics-7bd7489957-5gm8c\" (UID: \"b9c973ab-4163-413a-8969-2947502f3592\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-5gm8c"
Oct 30 21:53:35.439136 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:35.439066    4412 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-27snf\" (UniqueName: \"kubernetes.io/projected/78d5206b-1dec-4c66-97e1-32b034ce3529-kube-api-access-27snf\") pod \"node-exporter-ckb8k\" (UID: \"78d5206b-1dec-4c66-97e1-32b034ce3529\") " pod="openshift-monitoring/node-exporter-ckb8k"
Oct 30 21:53:35.439136 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:35.439095    4412 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/78d5206b-1dec-4c66-97e1-32b034ce3529-node-exporter-wtmp\") pod \"node-exporter-ckb8k\" (UID: \"78d5206b-1dec-4c66-97e1-32b034ce3529\") " pod="openshift-monitoring/node-exporter-ckb8k"
Oct 30 21:53:35.439301 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:35.439146    4412 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/78d5206b-1dec-4c66-97e1-32b034ce3529-node-exporter-tls\") pod \"node-exporter-ckb8k\" (UID: \"78d5206b-1dec-4c66-97e1-32b034ce3529\") " pod="openshift-monitoring/node-exporter-ckb8k"
Oct 30 21:53:35.439301 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:35.439248    4412 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/78d5206b-1dec-4c66-97e1-32b034ce3529-sys\") pod \"node-exporter-ckb8k\" (UID: \"78d5206b-1dec-4c66-97e1-32b034ce3529\") " pod="openshift-monitoring/node-exporter-ckb8k"
Oct 30 21:53:35.439375 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:35.439298    4412 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/78d5206b-1dec-4c66-97e1-32b034ce3529-metrics-client-ca\") pod \"node-exporter-ckb8k\" (UID: \"78d5206b-1dec-4c66-97e1-32b034ce3529\") " pod="openshift-monitoring/node-exporter-ckb8k"
Oct 30 21:53:35.439375 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:35.439355    4412 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/78d5206b-1dec-4c66-97e1-32b034ce3529-root\") pod \"node-exporter-ckb8k\" (UID: \"78d5206b-1dec-4c66-97e1-32b034ce3529\") " pod="openshift-monitoring/node-exporter-ckb8k"
Oct 30 21:53:35.439452 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:35.439381    4412 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/78d5206b-1dec-4c66-97e1-32b034ce3529-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-ckb8k\" (UID: \"78d5206b-1dec-4c66-97e1-32b034ce3529\") " pod="openshift-monitoring/node-exporter-ckb8k"
Oct 30 21:53:35.439452 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:35.439411    4412 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/78d5206b-1dec-4c66-97e1-32b034ce3529-node-exporter-accelerators-collector-config\") pod \"node-exporter-ckb8k\" (UID: \"78d5206b-1dec-4c66-97e1-32b034ce3529\") " pod="openshift-monitoring/node-exporter-ckb8k"
Oct 30 21:53:35.439551 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:35.439466    4412 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/78d5206b-1dec-4c66-97e1-32b034ce3529-node-exporter-textfile\") pod \"node-exporter-ckb8k\" (UID: \"78d5206b-1dec-4c66-97e1-32b034ce3529\") " pod="openshift-monitoring/node-exporter-ckb8k"
Oct 30 21:53:35.461061 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:35.461038    4412 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Oct 30 21:53:35.461313 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:35.461293    4412 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Oct 30 21:53:35.461383 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:35.461337    4412 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Oct 30 21:53:35.461446 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:35.461413    4412 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-dq6s5\""
Oct 30 21:53:35.465517 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:35.465502    4412 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-5gm8c"
Oct 30 21:53:35.540573 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:35.540546    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/78d5206b-1dec-4c66-97e1-32b034ce3529-node-exporter-accelerators-collector-config\") pod \"node-exporter-ckb8k\" (UID: \"78d5206b-1dec-4c66-97e1-32b034ce3529\") " pod="openshift-monitoring/node-exporter-ckb8k"
Oct 30 21:53:35.540703 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:35.540582    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/78d5206b-1dec-4c66-97e1-32b034ce3529-node-exporter-textfile\") pod \"node-exporter-ckb8k\" (UID: \"78d5206b-1dec-4c66-97e1-32b034ce3529\") " pod="openshift-monitoring/node-exporter-ckb8k"
Oct 30 21:53:35.540703 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:35.540604    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-27snf\" (UniqueName: \"kubernetes.io/projected/78d5206b-1dec-4c66-97e1-32b034ce3529-kube-api-access-27snf\") pod \"node-exporter-ckb8k\" (UID: \"78d5206b-1dec-4c66-97e1-32b034ce3529\") " pod="openshift-monitoring/node-exporter-ckb8k"
Oct 30 21:53:35.540703 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:35.540624    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/78d5206b-1dec-4c66-97e1-32b034ce3529-node-exporter-wtmp\") pod \"node-exporter-ckb8k\" (UID: \"78d5206b-1dec-4c66-97e1-32b034ce3529\") " pod="openshift-monitoring/node-exporter-ckb8k"
Oct 30 21:53:35.540703 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:35.540654    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/78d5206b-1dec-4c66-97e1-32b034ce3529-node-exporter-tls\") pod \"node-exporter-ckb8k\" (UID: \"78d5206b-1dec-4c66-97e1-32b034ce3529\") " pod="openshift-monitoring/node-exporter-ckb8k"
Oct 30 21:53:35.540920 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:35.540787    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/78d5206b-1dec-4c66-97e1-32b034ce3529-node-exporter-wtmp\") pod \"node-exporter-ckb8k\" (UID: \"78d5206b-1dec-4c66-97e1-32b034ce3529\") " pod="openshift-monitoring/node-exporter-ckb8k"
Oct 30 21:53:35.540920 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:35.540838    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/78d5206b-1dec-4c66-97e1-32b034ce3529-sys\") pod \"node-exporter-ckb8k\" (UID: \"78d5206b-1dec-4c66-97e1-32b034ce3529\") " pod="openshift-monitoring/node-exporter-ckb8k"
Oct 30 21:53:35.540920 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:35.540872    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/78d5206b-1dec-4c66-97e1-32b034ce3529-metrics-client-ca\") pod \"node-exporter-ckb8k\" (UID: \"78d5206b-1dec-4c66-97e1-32b034ce3529\") " pod="openshift-monitoring/node-exporter-ckb8k"
Oct 30 21:53:35.540920 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:35.540917    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/78d5206b-1dec-4c66-97e1-32b034ce3529-root\") pod \"node-exporter-ckb8k\" (UID: \"78d5206b-1dec-4c66-97e1-32b034ce3529\") " pod="openshift-monitoring/node-exporter-ckb8k"
Oct 30 21:53:35.541132 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:35.540926    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/78d5206b-1dec-4c66-97e1-32b034ce3529-node-exporter-textfile\") pod \"node-exporter-ckb8k\" (UID: \"78d5206b-1dec-4c66-97e1-32b034ce3529\") " pod="openshift-monitoring/node-exporter-ckb8k"
Oct 30 21:53:35.541132 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:35.540939    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/78d5206b-1dec-4c66-97e1-32b034ce3529-sys\") pod \"node-exporter-ckb8k\" (UID: \"78d5206b-1dec-4c66-97e1-32b034ce3529\") " pod="openshift-monitoring/node-exporter-ckb8k"
Oct 30 21:53:35.541132 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:35.540943    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/78d5206b-1dec-4c66-97e1-32b034ce3529-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-ckb8k\" (UID: \"78d5206b-1dec-4c66-97e1-32b034ce3529\") " pod="openshift-monitoring/node-exporter-ckb8k"
Oct 30 21:53:35.541132 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:35.540983    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/78d5206b-1dec-4c66-97e1-32b034ce3529-root\") pod \"node-exporter-ckb8k\" (UID: \"78d5206b-1dec-4c66-97e1-32b034ce3529\") " pod="openshift-monitoring/node-exporter-ckb8k"
Oct 30 21:53:35.541341 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:35.541309    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/78d5206b-1dec-4c66-97e1-32b034ce3529-node-exporter-accelerators-collector-config\") pod \"node-exporter-ckb8k\" (UID: \"78d5206b-1dec-4c66-97e1-32b034ce3529\") " pod="openshift-monitoring/node-exporter-ckb8k"
Oct 30 21:53:35.541430 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:35.541411    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/78d5206b-1dec-4c66-97e1-32b034ce3529-metrics-client-ca\") pod \"node-exporter-ckb8k\" (UID: \"78d5206b-1dec-4c66-97e1-32b034ce3529\") " pod="openshift-monitoring/node-exporter-ckb8k"
Oct 30 21:53:35.543000 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:35.542984    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/78d5206b-1dec-4c66-97e1-32b034ce3529-node-exporter-tls\") pod \"node-exporter-ckb8k\" (UID: \"78d5206b-1dec-4c66-97e1-32b034ce3529\") " pod="openshift-monitoring/node-exporter-ckb8k"
Oct 30 21:53:35.543205 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:35.543189    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/78d5206b-1dec-4c66-97e1-32b034ce3529-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-ckb8k\" (UID: \"78d5206b-1dec-4c66-97e1-32b034ce3529\") " pod="openshift-monitoring/node-exporter-ckb8k"
Oct 30 21:53:35.598423 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:35.598393    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-27snf\" (UniqueName: \"kubernetes.io/projected/78d5206b-1dec-4c66-97e1-32b034ce3529-kube-api-access-27snf\") pod \"node-exporter-ckb8k\" (UID: \"78d5206b-1dec-4c66-97e1-32b034ce3529\") " pod="openshift-monitoring/node-exporter-ckb8k"
Oct 30 21:53:35.623240 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:35.623219    4412 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-ckb8k"
Oct 30 21:53:35.633074 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:53:35.633039    4412 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod78d5206b_1dec_4c66_97e1_32b034ce3529.slice/crio-82cf7912006aa409461775724042bdc94800e8e6f6644f2b1d1dbd1471c27c4b WatchSource:0}: Error finding container 82cf7912006aa409461775724042bdc94800e8e6f6644f2b1d1dbd1471c27c4b: Status 404 returned error can't find the container with id 82cf7912006aa409461775724042bdc94800e8e6f6644f2b1d1dbd1471c27c4b
Oct 30 21:53:35.770525 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:35.770496    4412 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-ckb8k" event={"ID":"78d5206b-1dec-4c66-97e1-32b034ce3529","Type":"ContainerStarted","Data":"82cf7912006aa409461775724042bdc94800e8e6f6644f2b1d1dbd1471c27c4b"}
Oct 30 21:53:35.882274 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:35.882244    4412 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-7bd7489957-5gm8c"]
Oct 30 21:53:35.885029 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:53:35.884999    4412 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb9c973ab_4163_413a_8969_2947502f3592.slice/crio-6bff48ba6ebc2fddace556e5d3d2c88efcacf3fb6b8aa84cb8e474e601945471 WatchSource:0}: Error finding container 6bff48ba6ebc2fddace556e5d3d2c88efcacf3fb6b8aa84cb8e474e601945471: Status 404 returned error can't find the container with id 6bff48ba6ebc2fddace556e5d3d2c88efcacf3fb6b8aa84cb8e474e601945471
Oct 30 21:53:36.775976 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:36.775929    4412 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-5gm8c" event={"ID":"b9c973ab-4163-413a-8969-2947502f3592","Type":"ContainerStarted","Data":"86099dffd25e7c91c7b2042eea5a432674eb5f98941b1dc36925a2010f30fea7"}
Oct 30 21:53:36.776412 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:36.776044    4412 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-5gm8c" event={"ID":"b9c973ab-4163-413a-8969-2947502f3592","Type":"ContainerStarted","Data":"97fc74e940ca2cebb4a488ce1f02ff71318b376d43a72c47d781d37843cdfe13"}
Oct 30 21:53:36.776412 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:36.776074    4412 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-5gm8c" event={"ID":"b9c973ab-4163-413a-8969-2947502f3592","Type":"ContainerStarted","Data":"6bff48ba6ebc2fddace556e5d3d2c88efcacf3fb6b8aa84cb8e474e601945471"}
Oct 30 21:53:37.593332 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:37.593301    4412 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Oct 30 21:53:37.595639 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:37.595619    4412 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:53:37.649749 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:37.649713    4412 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Oct 30 21:53:37.649749 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:37.649728    4412 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Oct 30 21:53:37.649952 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:37.649789    4412 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Oct 30 21:53:37.650147 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:37.650108    4412 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Oct 30 21:53:37.650147 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:37.650142    4412 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Oct 30 21:53:37.650274 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:37.650162    4412 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Oct 30 21:53:37.650274 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:37.650185    4412 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Oct 30 21:53:37.650274 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:37.650194    4412 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Oct 30 21:53:37.658616 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:37.658594    4412 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/92120790-479e-4c07-9572-034b9988f73f-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"92120790-479e-4c07-9572-034b9988f73f\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:53:37.658722 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:37.658626    4412 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/92120790-479e-4c07-9572-034b9988f73f-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"92120790-479e-4c07-9572-034b9988f73f\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:53:37.658722 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:37.658704    4412 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/92120790-479e-4c07-9572-034b9988f73f-config-volume\") pod \"alertmanager-main-0\" (UID: \"92120790-479e-4c07-9572-034b9988f73f\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:53:37.658815 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:37.658742    4412 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/92120790-479e-4c07-9572-034b9988f73f-config-out\") pod \"alertmanager-main-0\" (UID: \"92120790-479e-4c07-9572-034b9988f73f\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:53:37.658850 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:37.658827    4412 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/92120790-479e-4c07-9572-034b9988f73f-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"92120790-479e-4c07-9572-034b9988f73f\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:53:37.658889 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:37.658853    4412 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/92120790-479e-4c07-9572-034b9988f73f-web-config\") pod \"alertmanager-main-0\" (UID: \"92120790-479e-4c07-9572-034b9988f73f\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:53:37.658922 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:37.658892    4412 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/92120790-479e-4c07-9572-034b9988f73f-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"92120790-479e-4c07-9572-034b9988f73f\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:53:37.658952 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:37.658925    4412 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/92120790-479e-4c07-9572-034b9988f73f-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"92120790-479e-4c07-9572-034b9988f73f\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:53:37.658952 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:37.658944    4412 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/92120790-479e-4c07-9572-034b9988f73f-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"92120790-479e-4c07-9572-034b9988f73f\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:53:37.659010 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:37.658961    4412 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/92120790-479e-4c07-9572-034b9988f73f-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"92120790-479e-4c07-9572-034b9988f73f\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:53:37.659010 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:37.658990    4412 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/92120790-479e-4c07-9572-034b9988f73f-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"92120790-479e-4c07-9572-034b9988f73f\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:53:37.659090 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:37.659065    4412 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/92120790-479e-4c07-9572-034b9988f73f-tls-assets\") pod \"alertmanager-main-0\" (UID: \"92120790-479e-4c07-9572-034b9988f73f\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:53:37.659172 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:37.659157    4412 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gq64p\" (UniqueName: \"kubernetes.io/projected/92120790-479e-4c07-9572-034b9988f73f-kube-api-access-gq64p\") pod \"alertmanager-main-0\" (UID: \"92120790-479e-4c07-9572-034b9988f73f\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:53:37.751096 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:37.751033    4412 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Oct 30 21:53:37.751437 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:37.751420    4412 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-rps27\""
Oct 30 21:53:37.752240 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:37.752218    4412 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Oct 30 21:53:37.759793 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:37.759774    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/92120790-479e-4c07-9572-034b9988f73f-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"92120790-479e-4c07-9572-034b9988f73f\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:53:37.759880 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:37.759800    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/92120790-479e-4c07-9572-034b9988f73f-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"92120790-479e-4c07-9572-034b9988f73f\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:53:37.759880 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:37.759823    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/92120790-479e-4c07-9572-034b9988f73f-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"92120790-479e-4c07-9572-034b9988f73f\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:53:37.759880 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:37.759847    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/92120790-479e-4c07-9572-034b9988f73f-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"92120790-479e-4c07-9572-034b9988f73f\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:53:37.759984 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:37.759970    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/92120790-479e-4c07-9572-034b9988f73f-tls-assets\") pod \"alertmanager-main-0\" (UID: \"92120790-479e-4c07-9572-034b9988f73f\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:53:37.760035 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:37.760000    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gq64p\" (UniqueName: \"kubernetes.io/projected/92120790-479e-4c07-9572-034b9988f73f-kube-api-access-gq64p\") pod \"alertmanager-main-0\" (UID: \"92120790-479e-4c07-9572-034b9988f73f\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:53:37.760201 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:37.760180    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/92120790-479e-4c07-9572-034b9988f73f-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"92120790-479e-4c07-9572-034b9988f73f\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:53:37.760296 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:37.760231    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/92120790-479e-4c07-9572-034b9988f73f-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"92120790-479e-4c07-9572-034b9988f73f\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:53:37.760296 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:37.760266    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/92120790-479e-4c07-9572-034b9988f73f-config-volume\") pod \"alertmanager-main-0\" (UID: \"92120790-479e-4c07-9572-034b9988f73f\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:53:37.760296 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:37.760288    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/92120790-479e-4c07-9572-034b9988f73f-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"92120790-479e-4c07-9572-034b9988f73f\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:53:37.760474 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:37.760338    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/92120790-479e-4c07-9572-034b9988f73f-config-out\") pod \"alertmanager-main-0\" (UID: \"92120790-479e-4c07-9572-034b9988f73f\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:53:37.760474 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:37.760415    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/92120790-479e-4c07-9572-034b9988f73f-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"92120790-479e-4c07-9572-034b9988f73f\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:53:37.760474 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:37.760440    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/92120790-479e-4c07-9572-034b9988f73f-web-config\") pod \"alertmanager-main-0\" (UID: \"92120790-479e-4c07-9572-034b9988f73f\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:53:37.760474 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:37.760463    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/92120790-479e-4c07-9572-034b9988f73f-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"92120790-479e-4c07-9572-034b9988f73f\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:53:37.761266 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:37.760687    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/92120790-479e-4c07-9572-034b9988f73f-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"92120790-479e-4c07-9572-034b9988f73f\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:53:37.761373 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:37.761354    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/92120790-479e-4c07-9572-034b9988f73f-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"92120790-479e-4c07-9572-034b9988f73f\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:53:37.763286 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:37.763232    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/92120790-479e-4c07-9572-034b9988f73f-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"92120790-479e-4c07-9572-034b9988f73f\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:53:37.763383 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:37.763357    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/92120790-479e-4c07-9572-034b9988f73f-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"92120790-479e-4c07-9572-034b9988f73f\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:53:37.763664 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:37.763615    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/92120790-479e-4c07-9572-034b9988f73f-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"92120790-479e-4c07-9572-034b9988f73f\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:53:37.763867 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:37.763851    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/92120790-479e-4c07-9572-034b9988f73f-config-volume\") pod \"alertmanager-main-0\" (UID: \"92120790-479e-4c07-9572-034b9988f73f\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:53:37.764030 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:37.764012    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/92120790-479e-4c07-9572-034b9988f73f-tls-assets\") pod \"alertmanager-main-0\" (UID: \"92120790-479e-4c07-9572-034b9988f73f\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:53:37.764135 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:37.764094    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/92120790-479e-4c07-9572-034b9988f73f-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"92120790-479e-4c07-9572-034b9988f73f\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:53:37.764376 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:37.764359    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/92120790-479e-4c07-9572-034b9988f73f-web-config\") pod \"alertmanager-main-0\" (UID: \"92120790-479e-4c07-9572-034b9988f73f\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:53:37.764456 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:37.764439    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/92120790-479e-4c07-9572-034b9988f73f-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"92120790-479e-4c07-9572-034b9988f73f\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:53:37.764871 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:37.764852    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/92120790-479e-4c07-9572-034b9988f73f-config-out\") pod \"alertmanager-main-0\" (UID: \"92120790-479e-4c07-9572-034b9988f73f\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:53:37.784264 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:37.784237    4412 generic.go:356] "Generic (PLEG): container finished" podID="78d5206b-1dec-4c66-97e1-32b034ce3529" containerID="44a095dc66079633a80c3e7ca8fbaa8f292173c89b921465db6951c1e0b36db3" exitCode=0
Oct 30 21:53:37.784628 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:37.784322    4412 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-ckb8k" event={"ID":"78d5206b-1dec-4c66-97e1-32b034ce3529","Type":"ContainerDied","Data":"44a095dc66079633a80c3e7ca8fbaa8f292173c89b921465db6951c1e0b36db3"}
Oct 30 21:53:37.786588 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:37.786554    4412 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-5gm8c" event={"ID":"b9c973ab-4163-413a-8969-2947502f3592","Type":"ContainerStarted","Data":"e000bbfc5f1a25b8eb3d46e63a03436004a7a70c515d1a2b360016fe046e5236"}
Oct 30 21:53:37.817983 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:37.817955    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gq64p\" (UniqueName: \"kubernetes.io/projected/92120790-479e-4c07-9572-034b9988f73f-kube-api-access-gq64p\") pod \"alertmanager-main-0\" (UID: \"92120790-479e-4c07-9572-034b9988f73f\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:53:37.911234 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:37.911211    4412 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:53:38.027429 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:38.027341    4412 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-5gm8c" podStartSLOduration=1.530237079 podStartE2EDuration="3.027327994s" totalImagesPullingTime="1.497090915s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:53:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:53:38.026163856 +0000 UTC m=+38.530835809" watchObservedRunningTime="2026-10-30 21:53:38.027327994 +0000 UTC m=+38.531999945"
Oct 30 21:53:38.343308 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:38.343184    4412 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Oct 30 21:53:38.345726 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:53:38.345700    4412 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod92120790_479e_4c07_9572_034b9988f73f.slice/crio-40dfc473b4111c8e69a651bf797fffbe928c17fc43a70e6ff1afc94df66f8688 WatchSource:0}: Error finding container 40dfc473b4111c8e69a651bf797fffbe928c17fc43a70e6ff1afc94df66f8688: Status 404 returned error can't find the container with id 40dfc473b4111c8e69a651bf797fffbe928c17fc43a70e6ff1afc94df66f8688
Oct 30 21:53:38.791641 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:38.791606    4412 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-ckb8k" event={"ID":"78d5206b-1dec-4c66-97e1-32b034ce3529","Type":"ContainerStarted","Data":"224f28c465c68920243ad5bd1657706bee4521f5a1dc3b9035bbcd51308e69b2"}
Oct 30 21:53:38.792088 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:38.791663    4412 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-ckb8k" event={"ID":"78d5206b-1dec-4c66-97e1-32b034ce3529","Type":"ContainerStarted","Data":"fe7924213c94f5ec1fd5d283e3ba4d34d4d4edf73f94c6e3f5e4ff64c51a9591"}
Oct 30 21:53:38.792829 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:38.792800    4412 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"92120790-479e-4c07-9572-034b9988f73f","Type":"ContainerStarted","Data":"40dfc473b4111c8e69a651bf797fffbe928c17fc43a70e6ff1afc94df66f8688"}
Oct 30 21:53:38.917240 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:38.917191    4412 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-ckb8k" podStartSLOduration=2.5323391219999998 podStartE2EDuration="3.917175452s" totalImagesPullingTime="1.301142261s" totalInitContainerRuntime="83.694069ms" isStatefulPod=true podCreationTimestamp="2026-10-30 21:53:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:53:38.916056137 +0000 UTC m=+39.420728089" watchObservedRunningTime="2026-10-30 21:53:38.917175452 +0000 UTC m=+39.421847405"
Oct 30 21:53:39.797325 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:39.797288    4412 generic.go:356] "Generic (PLEG): container finished" podID="92120790-479e-4c07-9572-034b9988f73f" containerID="cf3a5ed14c2e2e1af74992abbc7c89f830f6fd0d2da428bb55bdeae2bb456ba5" exitCode=0
Oct 30 21:53:39.797686 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:39.797389    4412 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"92120790-479e-4c07-9572-034b9988f73f","Type":"ContainerDied","Data":"cf3a5ed14c2e2e1af74992abbc7c89f830f6fd0d2da428bb55bdeae2bb456ba5"}
Oct 30 21:53:42.814002 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:42.813237    4412 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"92120790-479e-4c07-9572-034b9988f73f","Type":"ContainerStarted","Data":"e99719f4b0360f45c831aaa064966a4b0b15c748e53bbff1e8b2ebb545dee2cf"}
Oct 30 21:53:42.814002 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:42.813401    4412 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"92120790-479e-4c07-9572-034b9988f73f","Type":"ContainerStarted","Data":"ca19626bf72cc01e85fe0c5a1507b920da871704d01e0eb47b7136376b00176a"}
Oct 30 21:53:42.814002 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:42.813452    4412 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"92120790-479e-4c07-9572-034b9988f73f","Type":"ContainerStarted","Data":"a2ba4e25ee262a794c0a01941f9779e9739a6c810d8a0f6d43de098a190ae67a"}
Oct 30 21:53:42.814002 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:42.813502    4412 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"92120790-479e-4c07-9572-034b9988f73f","Type":"ContainerStarted","Data":"84427fc3a0bf3a71f656f122b23b836dbab3ad0277d8d1276ef3227fd3b16ffb"}
Oct 30 21:53:42.814002 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:42.813555    4412 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"92120790-479e-4c07-9572-034b9988f73f","Type":"ContainerStarted","Data":"6307c888d187ca8e02aa2411c1b2c26ede3009b02a67ca88f6e85f55f701e854"}
Oct 30 21:53:43.822972 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:43.822888    4412 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"92120790-479e-4c07-9572-034b9988f73f","Type":"ContainerStarted","Data":"26ea0ca411489e87eb970fae60a59161c575d7aeb3c8269a64d7e4653cf2bdd2"}
Oct 30 21:53:44.102807 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:44.102676    4412 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.361012692 podStartE2EDuration="7.102661478s" totalImagesPullingTime="4.728958263s" totalInitContainerRuntime="12.690523ms" isStatefulPod=false podCreationTimestamp="2026-10-30 21:53:37 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:53:44.079455323 +0000 UTC m=+44.584127274" watchObservedRunningTime="2026-10-30 21:53:44.102661478 +0000 UTC m=+44.607333428"
Oct 30 21:53:44.318395 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:44.318354    4412 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Oct 30 21:53:44.321572 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:44.321547    4412 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:53:44.379406 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:44.379337    4412 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Oct 30 21:53:44.380402 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:44.380375    4412 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Oct 30 21:53:44.417748 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:44.417718    4412 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/12507178-e97f-4c11-9a1b-1f398101d1ed-config-out\") pod \"prometheus-k8s-0\" (UID: \"12507178-e97f-4c11-9a1b-1f398101d1ed\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:53:44.417748 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:44.417747    4412 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/12507178-e97f-4c11-9a1b-1f398101d1ed-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"12507178-e97f-4c11-9a1b-1f398101d1ed\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:53:44.417908 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:44.417771    4412 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/12507178-e97f-4c11-9a1b-1f398101d1ed-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"12507178-e97f-4c11-9a1b-1f398101d1ed\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:53:44.417908 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:44.417789    4412 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/12507178-e97f-4c11-9a1b-1f398101d1ed-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"12507178-e97f-4c11-9a1b-1f398101d1ed\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:53:44.417973 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:44.417929    4412 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/12507178-e97f-4c11-9a1b-1f398101d1ed-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"12507178-e97f-4c11-9a1b-1f398101d1ed\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:53:44.417973 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:44.417946    4412 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/12507178-e97f-4c11-9a1b-1f398101d1ed-config\") pod \"prometheus-k8s-0\" (UID: \"12507178-e97f-4c11-9a1b-1f398101d1ed\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:53:44.417973 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:44.417962    4412 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/12507178-e97f-4c11-9a1b-1f398101d1ed-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"12507178-e97f-4c11-9a1b-1f398101d1ed\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:53:44.418055 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:44.417978    4412 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/12507178-e97f-4c11-9a1b-1f398101d1ed-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"12507178-e97f-4c11-9a1b-1f398101d1ed\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:53:44.418055 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:44.418023    4412 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/12507178-e97f-4c11-9a1b-1f398101d1ed-web-config\") pod \"prometheus-k8s-0\" (UID: \"12507178-e97f-4c11-9a1b-1f398101d1ed\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:53:44.418141 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:44.418057    4412 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/12507178-e97f-4c11-9a1b-1f398101d1ed-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"12507178-e97f-4c11-9a1b-1f398101d1ed\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:53:44.418141 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:44.418089    4412 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mn7ls\" (UniqueName: \"kubernetes.io/projected/12507178-e97f-4c11-9a1b-1f398101d1ed-kube-api-access-mn7ls\") pod \"prometheus-k8s-0\" (UID: \"12507178-e97f-4c11-9a1b-1f398101d1ed\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:53:44.418141 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:44.418110    4412 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/12507178-e97f-4c11-9a1b-1f398101d1ed-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"12507178-e97f-4c11-9a1b-1f398101d1ed\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:53:44.418232 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:44.418141    4412 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/12507178-e97f-4c11-9a1b-1f398101d1ed-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"12507178-e97f-4c11-9a1b-1f398101d1ed\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:53:44.418232 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:44.418211    4412 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/12507178-e97f-4c11-9a1b-1f398101d1ed-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"12507178-e97f-4c11-9a1b-1f398101d1ed\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:53:44.418291 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:44.418240    4412 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/12507178-e97f-4c11-9a1b-1f398101d1ed-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"12507178-e97f-4c11-9a1b-1f398101d1ed\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:53:44.418291 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:44.418262    4412 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/12507178-e97f-4c11-9a1b-1f398101d1ed-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"12507178-e97f-4c11-9a1b-1f398101d1ed\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:53:44.418345 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:44.418289    4412 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/12507178-e97f-4c11-9a1b-1f398101d1ed-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"12507178-e97f-4c11-9a1b-1f398101d1ed\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:53:44.418345 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:44.418308    4412 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/12507178-e97f-4c11-9a1b-1f398101d1ed-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"12507178-e97f-4c11-9a1b-1f398101d1ed\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:53:44.418345 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:44.418326    4412 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/12507178-e97f-4c11-9a1b-1f398101d1ed-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"12507178-e97f-4c11-9a1b-1f398101d1ed\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:53:44.418461 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:44.418347    4412 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/12507178-e97f-4c11-9a1b-1f398101d1ed-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"12507178-e97f-4c11-9a1b-1f398101d1ed\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:53:44.431905 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:44.431883    4412 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Oct 30 21:53:44.432192 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:44.432105    4412 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Oct 30 21:53:44.432192 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:44.432138    4412 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Oct 30 21:53:44.432192 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:44.432163    4412 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Oct 30 21:53:44.432192 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:44.432174    4412 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Oct 30 21:53:44.432192 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:44.432187    4412 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Oct 30 21:53:44.432467 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:44.432232    4412 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Oct 30 21:53:44.432467 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:44.432244    4412 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Oct 30 21:53:44.432467 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:44.432268    4412 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Oct 30 21:53:44.432467 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:44.432271    4412 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-f513mko3c0tve\""
Oct 30 21:53:44.432467 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:44.432321    4412 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-jfxm4\""
Oct 30 21:53:44.432467 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:44.432398    4412 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Oct 30 21:53:44.433533 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:44.433517    4412 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Oct 30 21:53:44.480260 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:44.480237    4412 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Oct 30 21:53:44.480535 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:44.480519    4412 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Oct 30 21:53:44.482420 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:44.482403    4412 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Oct 30 21:53:44.519270 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:44.519244    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/12507178-e97f-4c11-9a1b-1f398101d1ed-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"12507178-e97f-4c11-9a1b-1f398101d1ed\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:53:44.519387 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:44.519274    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/12507178-e97f-4c11-9a1b-1f398101d1ed-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"12507178-e97f-4c11-9a1b-1f398101d1ed\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:53:44.519387 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:44.519300    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/12507178-e97f-4c11-9a1b-1f398101d1ed-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"12507178-e97f-4c11-9a1b-1f398101d1ed\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:53:44.519387 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:44.519329    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/12507178-e97f-4c11-9a1b-1f398101d1ed-config\") pod \"prometheus-k8s-0\" (UID: \"12507178-e97f-4c11-9a1b-1f398101d1ed\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:53:44.519387 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:44.519351    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/12507178-e97f-4c11-9a1b-1f398101d1ed-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"12507178-e97f-4c11-9a1b-1f398101d1ed\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:53:44.519574 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:44.519391    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/12507178-e97f-4c11-9a1b-1f398101d1ed-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"12507178-e97f-4c11-9a1b-1f398101d1ed\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:53:44.519574 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:44.519418    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/12507178-e97f-4c11-9a1b-1f398101d1ed-web-config\") pod \"prometheus-k8s-0\" (UID: \"12507178-e97f-4c11-9a1b-1f398101d1ed\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:53:44.519574 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:44.519444    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/12507178-e97f-4c11-9a1b-1f398101d1ed-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"12507178-e97f-4c11-9a1b-1f398101d1ed\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:53:44.519574 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:44.519480    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mn7ls\" (UniqueName: \"kubernetes.io/projected/12507178-e97f-4c11-9a1b-1f398101d1ed-kube-api-access-mn7ls\") pod \"prometheus-k8s-0\" (UID: \"12507178-e97f-4c11-9a1b-1f398101d1ed\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:53:44.519574 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:44.519518    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/12507178-e97f-4c11-9a1b-1f398101d1ed-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"12507178-e97f-4c11-9a1b-1f398101d1ed\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:53:44.519574 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:44.519549    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/12507178-e97f-4c11-9a1b-1f398101d1ed-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"12507178-e97f-4c11-9a1b-1f398101d1ed\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:53:44.520311 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:44.519601    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/12507178-e97f-4c11-9a1b-1f398101d1ed-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"12507178-e97f-4c11-9a1b-1f398101d1ed\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:53:44.520311 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:44.519635    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/12507178-e97f-4c11-9a1b-1f398101d1ed-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"12507178-e97f-4c11-9a1b-1f398101d1ed\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:53:44.520311 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:44.519659    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/12507178-e97f-4c11-9a1b-1f398101d1ed-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"12507178-e97f-4c11-9a1b-1f398101d1ed\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:53:44.520311 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:44.519685    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/12507178-e97f-4c11-9a1b-1f398101d1ed-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"12507178-e97f-4c11-9a1b-1f398101d1ed\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:53:44.520311 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:44.519707    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/12507178-e97f-4c11-9a1b-1f398101d1ed-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"12507178-e97f-4c11-9a1b-1f398101d1ed\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:53:44.520311 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:44.519740    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/12507178-e97f-4c11-9a1b-1f398101d1ed-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"12507178-e97f-4c11-9a1b-1f398101d1ed\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:53:44.520311 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:44.519776    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/12507178-e97f-4c11-9a1b-1f398101d1ed-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"12507178-e97f-4c11-9a1b-1f398101d1ed\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:53:44.520311 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:44.519812    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/12507178-e97f-4c11-9a1b-1f398101d1ed-config-out\") pod \"prometheus-k8s-0\" (UID: \"12507178-e97f-4c11-9a1b-1f398101d1ed\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:53:44.520311 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:44.519838    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/12507178-e97f-4c11-9a1b-1f398101d1ed-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"12507178-e97f-4c11-9a1b-1f398101d1ed\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:53:44.520311 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:44.520201    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/12507178-e97f-4c11-9a1b-1f398101d1ed-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"12507178-e97f-4c11-9a1b-1f398101d1ed\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:53:44.520758 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:44.520394    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/12507178-e97f-4c11-9a1b-1f398101d1ed-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"12507178-e97f-4c11-9a1b-1f398101d1ed\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:53:44.523173 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:44.523147    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/12507178-e97f-4c11-9a1b-1f398101d1ed-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"12507178-e97f-4c11-9a1b-1f398101d1ed\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:53:44.523334 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:44.523142    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/12507178-e97f-4c11-9a1b-1f398101d1ed-web-config\") pod \"prometheus-k8s-0\" (UID: \"12507178-e97f-4c11-9a1b-1f398101d1ed\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:53:44.523458 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:44.523439    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/12507178-e97f-4c11-9a1b-1f398101d1ed-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"12507178-e97f-4c11-9a1b-1f398101d1ed\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:53:44.523861 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:44.523826    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/12507178-e97f-4c11-9a1b-1f398101d1ed-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"12507178-e97f-4c11-9a1b-1f398101d1ed\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:53:44.524457 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:44.524433    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/12507178-e97f-4c11-9a1b-1f398101d1ed-config\") pod \"prometheus-k8s-0\" (UID: \"12507178-e97f-4c11-9a1b-1f398101d1ed\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:53:44.524670 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:44.524648    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/12507178-e97f-4c11-9a1b-1f398101d1ed-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"12507178-e97f-4c11-9a1b-1f398101d1ed\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:53:44.530029 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:44.525763    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/12507178-e97f-4c11-9a1b-1f398101d1ed-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"12507178-e97f-4c11-9a1b-1f398101d1ed\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:53:44.530029 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:44.525997    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/12507178-e97f-4c11-9a1b-1f398101d1ed-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"12507178-e97f-4c11-9a1b-1f398101d1ed\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:53:44.530029 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:44.526280    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/12507178-e97f-4c11-9a1b-1f398101d1ed-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"12507178-e97f-4c11-9a1b-1f398101d1ed\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:53:44.530029 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:44.526571    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/12507178-e97f-4c11-9a1b-1f398101d1ed-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"12507178-e97f-4c11-9a1b-1f398101d1ed\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:53:44.530029 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:44.526842    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/12507178-e97f-4c11-9a1b-1f398101d1ed-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"12507178-e97f-4c11-9a1b-1f398101d1ed\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:53:44.530029 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:44.526984    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/12507178-e97f-4c11-9a1b-1f398101d1ed-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"12507178-e97f-4c11-9a1b-1f398101d1ed\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:53:44.530029 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:44.527913    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/12507178-e97f-4c11-9a1b-1f398101d1ed-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"12507178-e97f-4c11-9a1b-1f398101d1ed\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:53:44.530575 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:44.530555    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/12507178-e97f-4c11-9a1b-1f398101d1ed-config-out\") pod \"prometheus-k8s-0\" (UID: \"12507178-e97f-4c11-9a1b-1f398101d1ed\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:53:44.530677 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:44.530659    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/12507178-e97f-4c11-9a1b-1f398101d1ed-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"12507178-e97f-4c11-9a1b-1f398101d1ed\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:53:44.530714 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:44.530693    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/12507178-e97f-4c11-9a1b-1f398101d1ed-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"12507178-e97f-4c11-9a1b-1f398101d1ed\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:53:44.531761 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:44.531739    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/12507178-e97f-4c11-9a1b-1f398101d1ed-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"12507178-e97f-4c11-9a1b-1f398101d1ed\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:53:44.578616 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:44.578592    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mn7ls\" (UniqueName: \"kubernetes.io/projected/12507178-e97f-4c11-9a1b-1f398101d1ed-kube-api-access-mn7ls\") pod \"prometheus-k8s-0\" (UID: \"12507178-e97f-4c11-9a1b-1f398101d1ed\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:53:44.642883 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:44.642820    4412 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:53:45.076858 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:45.076753    4412 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Oct 30 21:53:45.079401 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:53:45.079374    4412 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod12507178_e97f_4c11_9a1b_1f398101d1ed.slice/crio-3102a387134ebbcddc6a1f857287a0069a6b8eadf4f3eb4343aac61b5d152e1e WatchSource:0}: Error finding container 3102a387134ebbcddc6a1f857287a0069a6b8eadf4f3eb4343aac61b5d152e1e: Status 404 returned error can't find the container with id 3102a387134ebbcddc6a1f857287a0069a6b8eadf4f3eb4343aac61b5d152e1e
Oct 30 21:53:45.735525 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:45.735499    4412 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-54cc95575d-b877n"
Oct 30 21:53:45.831222 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:45.831189    4412 generic.go:356] "Generic (PLEG): container finished" podID="12507178-e97f-4c11-9a1b-1f398101d1ed" containerID="cfb715bdde296b49e62e8d3dc1680612dad490050b54bce29720b044592a9054" exitCode=0
Oct 30 21:53:45.831385 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:45.831284    4412 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"12507178-e97f-4c11-9a1b-1f398101d1ed","Type":"ContainerDied","Data":"cfb715bdde296b49e62e8d3dc1680612dad490050b54bce29720b044592a9054"}
Oct 30 21:53:45.831385 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:45.831372    4412 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"12507178-e97f-4c11-9a1b-1f398101d1ed","Type":"ContainerStarted","Data":"3102a387134ebbcddc6a1f857287a0069a6b8eadf4f3eb4343aac61b5d152e1e"}
Oct 30 21:53:49.847182 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:49.847148    4412 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"12507178-e97f-4c11-9a1b-1f398101d1ed","Type":"ContainerStarted","Data":"583cf6620b731785d813dc08e3347f393468afe31c01671a5d09b8cd7d35fdfa"}
Oct 30 21:53:49.847602 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:49.847216    4412 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"12507178-e97f-4c11-9a1b-1f398101d1ed","Type":"ContainerStarted","Data":"20160ccf3bff21dff5b3b07c02829aada5e4b2b6b75d8715957176258b975205"}
Oct 30 21:53:52.860715 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:52.860679    4412 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"12507178-e97f-4c11-9a1b-1f398101d1ed","Type":"ContainerStarted","Data":"7d47b973d606a4331f6014c56255bbf9594fd1d3ddf41e42e9b1c176bae99d26"}
Oct 30 21:53:52.861201 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:52.860765    4412 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"12507178-e97f-4c11-9a1b-1f398101d1ed","Type":"ContainerStarted","Data":"8751397d963b2f8b29d7a15e3df56dd34a68504efb93929a396fa224b48dc3fe"}
Oct 30 21:53:52.861201 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:52.860792    4412 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"12507178-e97f-4c11-9a1b-1f398101d1ed","Type":"ContainerStarted","Data":"5cab152d119b329ca208021d7239b647136dcf716754c8ac35e9f5f54b8d49d1"}
Oct 30 21:53:52.861201 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:52.860813    4412 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"12507178-e97f-4c11-9a1b-1f398101d1ed","Type":"ContainerStarted","Data":"1a1379b25dc909eb29ef51c8fadb581d667bbde25e26c978f2d42691f1c4eee0"}
Oct 30 21:53:53.004651 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:53.004601    4412 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.992058827 podStartE2EDuration="9.004584153s" totalImagesPullingTime="5.99695765s" totalInitContainerRuntime="15.567676ms" isStatefulPod=false podCreationTimestamp="2026-10-30 21:53:44 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:53:53.003999409 +0000 UTC m=+53.508671361" watchObservedRunningTime="2026-10-30 21:53:53.004584153 +0000 UTC m=+53.509256146"
Oct 30 21:53:54.643827 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:53:54.643794    4412 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:54:04.448788 ip-10-0-131-202 kubenswrapper[4412]: E1030 21:54:04.448752    4412 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bkhmz" containerName="extractor"
Oct 30 21:54:34.448650 ip-10-0-131-202 kubenswrapper[4412]: E1030 21:54:34.448601    4412 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bkhmz" containerName="extractor"
Oct 30 21:54:51.488803 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:54:51.488760    4412 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:54:51.507307 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:54:51.507282    4412 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:54:52.073372 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:54:52.073346    4412 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:55:01.236174 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:01.236139    4412 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Oct 30 21:55:01.236571 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:01.236533    4412 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="92120790-479e-4c07-9572-034b9988f73f" containerName="alertmanager" containerID="cri-o://6307c888d187ca8e02aa2411c1b2c26ede3009b02a67ca88f6e85f55f701e854" gracePeriod=120
Oct 30 21:55:01.236622 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:01.236552    4412 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="92120790-479e-4c07-9572-034b9988f73f" containerName="kube-rbac-proxy-metric" containerID="cri-o://e99719f4b0360f45c831aaa064966a4b0b15c748e53bbff1e8b2ebb545dee2cf" gracePeriod=120
Oct 30 21:55:01.236622 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:01.236572    4412 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="92120790-479e-4c07-9572-034b9988f73f" containerName="kube-rbac-proxy" containerID="cri-o://ca19626bf72cc01e85fe0c5a1507b920da871704d01e0eb47b7136376b00176a" gracePeriod=120
Oct 30 21:55:01.236695 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:01.236628    4412 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="92120790-479e-4c07-9572-034b9988f73f" containerName="config-reloader" containerID="cri-o://84427fc3a0bf3a71f656f122b23b836dbab3ad0277d8d1276ef3227fd3b16ffb" gracePeriod=120
Oct 30 21:55:01.236695 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:01.236626    4412 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="92120790-479e-4c07-9572-034b9988f73f" containerName="prom-label-proxy" containerID="cri-o://26ea0ca411489e87eb970fae60a59161c575d7aeb3c8269a64d7e4653cf2bdd2" gracePeriod=120
Oct 30 21:55:01.236695 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:01.236552    4412 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="92120790-479e-4c07-9572-034b9988f73f" containerName="kube-rbac-proxy-web" containerID="cri-o://a2ba4e25ee262a794c0a01941f9779e9739a6c810d8a0f6d43de098a190ae67a" gracePeriod=120
Oct 30 21:55:02.096076 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:02.096031    4412 generic.go:356] "Generic (PLEG): container finished" podID="92120790-479e-4c07-9572-034b9988f73f" containerID="26ea0ca411489e87eb970fae60a59161c575d7aeb3c8269a64d7e4653cf2bdd2" exitCode=0
Oct 30 21:55:02.096076 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:02.096068    4412 generic.go:356] "Generic (PLEG): container finished" podID="92120790-479e-4c07-9572-034b9988f73f" containerID="e99719f4b0360f45c831aaa064966a4b0b15c748e53bbff1e8b2ebb545dee2cf" exitCode=0
Oct 30 21:55:02.096076 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:02.096078    4412 generic.go:356] "Generic (PLEG): container finished" podID="92120790-479e-4c07-9572-034b9988f73f" containerID="ca19626bf72cc01e85fe0c5a1507b920da871704d01e0eb47b7136376b00176a" exitCode=0
Oct 30 21:55:02.096339 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:02.096086    4412 generic.go:356] "Generic (PLEG): container finished" podID="92120790-479e-4c07-9572-034b9988f73f" containerID="84427fc3a0bf3a71f656f122b23b836dbab3ad0277d8d1276ef3227fd3b16ffb" exitCode=0
Oct 30 21:55:02.096339 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:02.096095    4412 generic.go:356] "Generic (PLEG): container finished" podID="92120790-479e-4c07-9572-034b9988f73f" containerID="6307c888d187ca8e02aa2411c1b2c26ede3009b02a67ca88f6e85f55f701e854" exitCode=0
Oct 30 21:55:02.096339 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:02.096108    4412 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"92120790-479e-4c07-9572-034b9988f73f","Type":"ContainerDied","Data":"26ea0ca411489e87eb970fae60a59161c575d7aeb3c8269a64d7e4653cf2bdd2"}
Oct 30 21:55:02.096339 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:02.096192    4412 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"92120790-479e-4c07-9572-034b9988f73f","Type":"ContainerDied","Data":"e99719f4b0360f45c831aaa064966a4b0b15c748e53bbff1e8b2ebb545dee2cf"}
Oct 30 21:55:02.096339 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:02.096220    4412 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"92120790-479e-4c07-9572-034b9988f73f","Type":"ContainerDied","Data":"ca19626bf72cc01e85fe0c5a1507b920da871704d01e0eb47b7136376b00176a"}
Oct 30 21:55:02.096339 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:02.096245    4412 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"92120790-479e-4c07-9572-034b9988f73f","Type":"ContainerDied","Data":"84427fc3a0bf3a71f656f122b23b836dbab3ad0277d8d1276ef3227fd3b16ffb"}
Oct 30 21:55:02.096339 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:02.096267    4412 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"92120790-479e-4c07-9572-034b9988f73f","Type":"ContainerDied","Data":"6307c888d187ca8e02aa2411c1b2c26ede3009b02a67ca88f6e85f55f701e854"}
Oct 30 21:55:02.497498 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:02.497470    4412 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:55:02.574038 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:02.574012    4412 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/92120790-479e-4c07-9572-034b9988f73f-tls-assets\" (UniqueName: \"kubernetes.io/projected/92120790-479e-4c07-9572-034b9988f73f-tls-assets\") pod \"92120790-479e-4c07-9572-034b9988f73f\" (UID: \"92120790-479e-4c07-9572-034b9988f73f\") "
Oct 30 21:55:02.574177 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:02.574060    4412 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/92120790-479e-4c07-9572-034b9988f73f-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/92120790-479e-4c07-9572-034b9988f73f-metrics-client-ca\") pod \"92120790-479e-4c07-9572-034b9988f73f\" (UID: \"92120790-479e-4c07-9572-034b9988f73f\") "
Oct 30 21:55:02.574177 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:02.574096    4412 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/92120790-479e-4c07-9572-034b9988f73f-web-config\" (UniqueName: \"kubernetes.io/secret/92120790-479e-4c07-9572-034b9988f73f-web-config\") pod \"92120790-479e-4c07-9572-034b9988f73f\" (UID: \"92120790-479e-4c07-9572-034b9988f73f\") "
Oct 30 21:55:02.574177 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:02.574152    4412 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/92120790-479e-4c07-9572-034b9988f73f-secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/92120790-479e-4c07-9572-034b9988f73f-secret-alertmanager-main-tls\") pod \"92120790-479e-4c07-9572-034b9988f73f\" (UID: \"92120790-479e-4c07-9572-034b9988f73f\") "
Oct 30 21:55:02.574318 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:02.574189    4412 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/92120790-479e-4c07-9572-034b9988f73f-alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/92120790-479e-4c07-9572-034b9988f73f-alertmanager-main-db\") pod \"92120790-479e-4c07-9572-034b9988f73f\" (UID: \"92120790-479e-4c07-9572-034b9988f73f\") "
Oct 30 21:55:02.574318 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:02.574224    4412 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/92120790-479e-4c07-9572-034b9988f73f-secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/92120790-479e-4c07-9572-034b9988f73f-secret-alertmanager-kube-rbac-proxy\") pod \"92120790-479e-4c07-9572-034b9988f73f\" (UID: \"92120790-479e-4c07-9572-034b9988f73f\") "
Oct 30 21:55:02.574318 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:02.574250    4412 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/92120790-479e-4c07-9572-034b9988f73f-cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/92120790-479e-4c07-9572-034b9988f73f-cluster-tls-config\") pod \"92120790-479e-4c07-9572-034b9988f73f\" (UID: \"92120790-479e-4c07-9572-034b9988f73f\") "
Oct 30 21:55:02.574318 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:02.574297    4412 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/92120790-479e-4c07-9572-034b9988f73f-config-volume\" (UniqueName: \"kubernetes.io/secret/92120790-479e-4c07-9572-034b9988f73f-config-volume\") pod \"92120790-479e-4c07-9572-034b9988f73f\" (UID: \"92120790-479e-4c07-9572-034b9988f73f\") "
Oct 30 21:55:02.574502 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:02.574324    4412 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/92120790-479e-4c07-9572-034b9988f73f-kube-api-access-gq64p\" (UniqueName: \"kubernetes.io/projected/92120790-479e-4c07-9572-034b9988f73f-kube-api-access-gq64p\") pod \"92120790-479e-4c07-9572-034b9988f73f\" (UID: \"92120790-479e-4c07-9572-034b9988f73f\") "
Oct 30 21:55:02.574502 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:02.574360    4412 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/92120790-479e-4c07-9572-034b9988f73f-secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/92120790-479e-4c07-9572-034b9988f73f-secret-alertmanager-kube-rbac-proxy-metric\") pod \"92120790-479e-4c07-9572-034b9988f73f\" (UID: \"92120790-479e-4c07-9572-034b9988f73f\") "
Oct 30 21:55:02.574502 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:02.574396    4412 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/92120790-479e-4c07-9572-034b9988f73f-secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/92120790-479e-4c07-9572-034b9988f73f-secret-alertmanager-kube-rbac-proxy-web\") pod \"92120790-479e-4c07-9572-034b9988f73f\" (UID: \"92120790-479e-4c07-9572-034b9988f73f\") "
Oct 30 21:55:02.574502 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:02.574442    4412 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/92120790-479e-4c07-9572-034b9988f73f-alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/92120790-479e-4c07-9572-034b9988f73f-alertmanager-trusted-ca-bundle\") pod \"92120790-479e-4c07-9572-034b9988f73f\" (UID: \"92120790-479e-4c07-9572-034b9988f73f\") "
Oct 30 21:55:02.574502 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:02.574447    4412 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/92120790-479e-4c07-9572-034b9988f73f-metrics-client-ca" pod "92120790-479e-4c07-9572-034b9988f73f" (UID: "92120790-479e-4c07-9572-034b9988f73f"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:55:02.574502 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:02.574476    4412 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/92120790-479e-4c07-9572-034b9988f73f-config-out\" (UniqueName: \"kubernetes.io/empty-dir/92120790-479e-4c07-9572-034b9988f73f-config-out\") pod \"92120790-479e-4c07-9572-034b9988f73f\" (UID: \"92120790-479e-4c07-9572-034b9988f73f\") "
Oct 30 21:55:02.574958 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:02.574694    4412 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/92120790-479e-4c07-9572-034b9988f73f-metrics-client-ca\") on node \"ip-10-0-131-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:55:02.575058 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:02.575005    4412 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/92120790-479e-4c07-9572-034b9988f73f-alertmanager-main-db" pod "92120790-479e-4c07-9572-034b9988f73f" (UID: "92120790-479e-4c07-9572-034b9988f73f"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:55:02.575590 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:02.575557    4412 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/92120790-479e-4c07-9572-034b9988f73f-alertmanager-trusted-ca-bundle" pod "92120790-479e-4c07-9572-034b9988f73f" (UID: "92120790-479e-4c07-9572-034b9988f73f"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:55:02.577452 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:02.577418    4412 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/92120790-479e-4c07-9572-034b9988f73f-secret-alertmanager-main-tls" pod "92120790-479e-4c07-9572-034b9988f73f" (UID: "92120790-479e-4c07-9572-034b9988f73f"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:55:02.577807 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:02.577772    4412 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/92120790-479e-4c07-9572-034b9988f73f-tls-assets" pod "92120790-479e-4c07-9572-034b9988f73f" (UID: "92120790-479e-4c07-9572-034b9988f73f"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:55:02.578066 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:02.578042    4412 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/92120790-479e-4c07-9572-034b9988f73f-secret-alertmanager-kube-rbac-proxy-web" pod "92120790-479e-4c07-9572-034b9988f73f" (UID: "92120790-479e-4c07-9572-034b9988f73f"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:55:02.578174 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:02.578158    4412 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/92120790-479e-4c07-9572-034b9988f73f-config-out" pod "92120790-479e-4c07-9572-034b9988f73f" (UID: "92120790-479e-4c07-9572-034b9988f73f"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:55:02.578337 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:02.578313    4412 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/92120790-479e-4c07-9572-034b9988f73f-kube-api-access-gq64p" pod "92120790-479e-4c07-9572-034b9988f73f" (UID: "92120790-479e-4c07-9572-034b9988f73f"). InnerVolumeSpecName "kube-api-access-gq64p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:55:02.578661 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:02.578622    4412 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/92120790-479e-4c07-9572-034b9988f73f-secret-alertmanager-kube-rbac-proxy-metric" pod "92120790-479e-4c07-9572-034b9988f73f" (UID: "92120790-479e-4c07-9572-034b9988f73f"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:55:02.578756 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:02.578718    4412 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/92120790-479e-4c07-9572-034b9988f73f-secret-alertmanager-kube-rbac-proxy" pod "92120790-479e-4c07-9572-034b9988f73f" (UID: "92120790-479e-4c07-9572-034b9988f73f"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:55:02.579049 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:02.579026    4412 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/92120790-479e-4c07-9572-034b9988f73f-config-volume" pod "92120790-479e-4c07-9572-034b9988f73f" (UID: "92120790-479e-4c07-9572-034b9988f73f"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:55:02.582924 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:02.582901    4412 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/92120790-479e-4c07-9572-034b9988f73f-cluster-tls-config" pod "92120790-479e-4c07-9572-034b9988f73f" (UID: "92120790-479e-4c07-9572-034b9988f73f"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:55:02.588914 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:02.588890    4412 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/92120790-479e-4c07-9572-034b9988f73f-web-config" pod "92120790-479e-4c07-9572-034b9988f73f" (UID: "92120790-479e-4c07-9572-034b9988f73f"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:55:02.675012 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:02.674960    4412 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/92120790-479e-4c07-9572-034b9988f73f-alertmanager-main-db\") on node \"ip-10-0-131-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:55:02.675012 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:02.674984    4412 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/92120790-479e-4c07-9572-034b9988f73f-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-131-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:55:02.675012 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:02.674995    4412 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/92120790-479e-4c07-9572-034b9988f73f-cluster-tls-config\") on node \"ip-10-0-131-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:55:02.675012 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:02.675005    4412 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/92120790-479e-4c07-9572-034b9988f73f-config-volume\") on node \"ip-10-0-131-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:55:02.675012 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:02.675013    4412 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gq64p\" (UniqueName: \"kubernetes.io/projected/92120790-479e-4c07-9572-034b9988f73f-kube-api-access-gq64p\") on node \"ip-10-0-131-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:55:02.675266 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:02.675023    4412 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/92120790-479e-4c07-9572-034b9988f73f-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-131-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:55:02.675266 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:02.675032    4412 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/92120790-479e-4c07-9572-034b9988f73f-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-131-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:55:02.675266 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:02.675040    4412 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/92120790-479e-4c07-9572-034b9988f73f-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-131-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:55:02.675266 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:02.675049    4412 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/92120790-479e-4c07-9572-034b9988f73f-config-out\") on node \"ip-10-0-131-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:55:02.675266 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:02.675057    4412 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/92120790-479e-4c07-9572-034b9988f73f-tls-assets\") on node \"ip-10-0-131-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:55:02.675266 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:02.675065    4412 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/92120790-479e-4c07-9572-034b9988f73f-web-config\") on node \"ip-10-0-131-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:55:02.675266 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:02.675072    4412 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/92120790-479e-4c07-9572-034b9988f73f-secret-alertmanager-main-tls\") on node \"ip-10-0-131-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:55:03.102762 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:03.102727    4412 generic.go:356] "Generic (PLEG): container finished" podID="92120790-479e-4c07-9572-034b9988f73f" containerID="a2ba4e25ee262a794c0a01941f9779e9739a6c810d8a0f6d43de098a190ae67a" exitCode=0
Oct 30 21:55:03.102938 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:03.102809    4412 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:55:03.102938 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:03.102802    4412 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"92120790-479e-4c07-9572-034b9988f73f","Type":"ContainerDied","Data":"a2ba4e25ee262a794c0a01941f9779e9739a6c810d8a0f6d43de098a190ae67a"}
Oct 30 21:55:03.103010 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:03.102941    4412 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"92120790-479e-4c07-9572-034b9988f73f","Type":"ContainerDied","Data":"40dfc473b4111c8e69a651bf797fffbe928c17fc43a70e6ff1afc94df66f8688"}
Oct 30 21:55:03.103010 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:03.102985    4412 scope.go:122] "RemoveContainer" containerID="26ea0ca411489e87eb970fae60a59161c575d7aeb3c8269a64d7e4653cf2bdd2"
Oct 30 21:55:03.110893 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:03.110878    4412 scope.go:122] "RemoveContainer" containerID="e99719f4b0360f45c831aaa064966a4b0b15c748e53bbff1e8b2ebb545dee2cf"
Oct 30 21:55:03.121730 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:03.121711    4412 scope.go:122] "RemoveContainer" containerID="ca19626bf72cc01e85fe0c5a1507b920da871704d01e0eb47b7136376b00176a"
Oct 30 21:55:03.128260 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:03.128237    4412 scope.go:122] "RemoveContainer" containerID="a2ba4e25ee262a794c0a01941f9779e9739a6c810d8a0f6d43de098a190ae67a"
Oct 30 21:55:03.134554 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:03.134533    4412 scope.go:122] "RemoveContainer" containerID="84427fc3a0bf3a71f656f122b23b836dbab3ad0277d8d1276ef3227fd3b16ffb"
Oct 30 21:55:03.141203 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:03.141186    4412 scope.go:122] "RemoveContainer" containerID="6307c888d187ca8e02aa2411c1b2c26ede3009b02a67ca88f6e85f55f701e854"
Oct 30 21:55:03.147739 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:03.147723    4412 scope.go:122] "RemoveContainer" containerID="cf3a5ed14c2e2e1af74992abbc7c89f830f6fd0d2da428bb55bdeae2bb456ba5"
Oct 30 21:55:03.154166 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:03.154145    4412 scope.go:122] "RemoveContainer" containerID="26ea0ca411489e87eb970fae60a59161c575d7aeb3c8269a64d7e4653cf2bdd2"
Oct 30 21:55:03.154446 ip-10-0-131-202 kubenswrapper[4412]: E1030 21:55:03.154426    4412 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"26ea0ca411489e87eb970fae60a59161c575d7aeb3c8269a64d7e4653cf2bdd2\": container with ID starting with 26ea0ca411489e87eb970fae60a59161c575d7aeb3c8269a64d7e4653cf2bdd2 not found: ID does not exist" containerID="26ea0ca411489e87eb970fae60a59161c575d7aeb3c8269a64d7e4653cf2bdd2"
Oct 30 21:55:03.154508 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:03.154456    4412 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"26ea0ca411489e87eb970fae60a59161c575d7aeb3c8269a64d7e4653cf2bdd2"} err="failed to get container status \"26ea0ca411489e87eb970fae60a59161c575d7aeb3c8269a64d7e4653cf2bdd2\": rpc error: code = NotFound desc = could not find container \"26ea0ca411489e87eb970fae60a59161c575d7aeb3c8269a64d7e4653cf2bdd2\": container with ID starting with 26ea0ca411489e87eb970fae60a59161c575d7aeb3c8269a64d7e4653cf2bdd2 not found: ID does not exist"
Oct 30 21:55:03.154508 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:03.154489    4412 scope.go:122] "RemoveContainer" containerID="e99719f4b0360f45c831aaa064966a4b0b15c748e53bbff1e8b2ebb545dee2cf"
Oct 30 21:55:03.154724 ip-10-0-131-202 kubenswrapper[4412]: E1030 21:55:03.154706    4412 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e99719f4b0360f45c831aaa064966a4b0b15c748e53bbff1e8b2ebb545dee2cf\": container with ID starting with e99719f4b0360f45c831aaa064966a4b0b15c748e53bbff1e8b2ebb545dee2cf not found: ID does not exist" containerID="e99719f4b0360f45c831aaa064966a4b0b15c748e53bbff1e8b2ebb545dee2cf"
Oct 30 21:55:03.154765 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:03.154727    4412 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e99719f4b0360f45c831aaa064966a4b0b15c748e53bbff1e8b2ebb545dee2cf"} err="failed to get container status \"e99719f4b0360f45c831aaa064966a4b0b15c748e53bbff1e8b2ebb545dee2cf\": rpc error: code = NotFound desc = could not find container \"e99719f4b0360f45c831aaa064966a4b0b15c748e53bbff1e8b2ebb545dee2cf\": container with ID starting with e99719f4b0360f45c831aaa064966a4b0b15c748e53bbff1e8b2ebb545dee2cf not found: ID does not exist"
Oct 30 21:55:03.154765 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:03.154739    4412 scope.go:122] "RemoveContainer" containerID="ca19626bf72cc01e85fe0c5a1507b920da871704d01e0eb47b7136376b00176a"
Oct 30 21:55:03.154961 ip-10-0-131-202 kubenswrapper[4412]: E1030 21:55:03.154944    4412 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ca19626bf72cc01e85fe0c5a1507b920da871704d01e0eb47b7136376b00176a\": container with ID starting with ca19626bf72cc01e85fe0c5a1507b920da871704d01e0eb47b7136376b00176a not found: ID does not exist" containerID="ca19626bf72cc01e85fe0c5a1507b920da871704d01e0eb47b7136376b00176a"
Oct 30 21:55:03.155038 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:03.154964    4412 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ca19626bf72cc01e85fe0c5a1507b920da871704d01e0eb47b7136376b00176a"} err="failed to get container status \"ca19626bf72cc01e85fe0c5a1507b920da871704d01e0eb47b7136376b00176a\": rpc error: code = NotFound desc = could not find container \"ca19626bf72cc01e85fe0c5a1507b920da871704d01e0eb47b7136376b00176a\": container with ID starting with ca19626bf72cc01e85fe0c5a1507b920da871704d01e0eb47b7136376b00176a not found: ID does not exist"
Oct 30 21:55:03.155038 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:03.154976    4412 scope.go:122] "RemoveContainer" containerID="a2ba4e25ee262a794c0a01941f9779e9739a6c810d8a0f6d43de098a190ae67a"
Oct 30 21:55:03.155174 ip-10-0-131-202 kubenswrapper[4412]: E1030 21:55:03.155158    4412 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a2ba4e25ee262a794c0a01941f9779e9739a6c810d8a0f6d43de098a190ae67a\": container with ID starting with a2ba4e25ee262a794c0a01941f9779e9739a6c810d8a0f6d43de098a190ae67a not found: ID does not exist" containerID="a2ba4e25ee262a794c0a01941f9779e9739a6c810d8a0f6d43de098a190ae67a"
Oct 30 21:55:03.155208 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:03.155177    4412 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a2ba4e25ee262a794c0a01941f9779e9739a6c810d8a0f6d43de098a190ae67a"} err="failed to get container status \"a2ba4e25ee262a794c0a01941f9779e9739a6c810d8a0f6d43de098a190ae67a\": rpc error: code = NotFound desc = could not find container \"a2ba4e25ee262a794c0a01941f9779e9739a6c810d8a0f6d43de098a190ae67a\": container with ID starting with a2ba4e25ee262a794c0a01941f9779e9739a6c810d8a0f6d43de098a190ae67a not found: ID does not exist"
Oct 30 21:55:03.155208 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:03.155188    4412 scope.go:122] "RemoveContainer" containerID="84427fc3a0bf3a71f656f122b23b836dbab3ad0277d8d1276ef3227fd3b16ffb"
Oct 30 21:55:03.155365 ip-10-0-131-202 kubenswrapper[4412]: E1030 21:55:03.155351    4412 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"84427fc3a0bf3a71f656f122b23b836dbab3ad0277d8d1276ef3227fd3b16ffb\": container with ID starting with 84427fc3a0bf3a71f656f122b23b836dbab3ad0277d8d1276ef3227fd3b16ffb not found: ID does not exist" containerID="84427fc3a0bf3a71f656f122b23b836dbab3ad0277d8d1276ef3227fd3b16ffb"
Oct 30 21:55:03.155404 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:03.155367    4412 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"84427fc3a0bf3a71f656f122b23b836dbab3ad0277d8d1276ef3227fd3b16ffb"} err="failed to get container status \"84427fc3a0bf3a71f656f122b23b836dbab3ad0277d8d1276ef3227fd3b16ffb\": rpc error: code = NotFound desc = could not find container \"84427fc3a0bf3a71f656f122b23b836dbab3ad0277d8d1276ef3227fd3b16ffb\": container with ID starting with 84427fc3a0bf3a71f656f122b23b836dbab3ad0277d8d1276ef3227fd3b16ffb not found: ID does not exist"
Oct 30 21:55:03.155404 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:03.155378    4412 scope.go:122] "RemoveContainer" containerID="6307c888d187ca8e02aa2411c1b2c26ede3009b02a67ca88f6e85f55f701e854"
Oct 30 21:55:03.155585 ip-10-0-131-202 kubenswrapper[4412]: E1030 21:55:03.155570    4412 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6307c888d187ca8e02aa2411c1b2c26ede3009b02a67ca88f6e85f55f701e854\": container with ID starting with 6307c888d187ca8e02aa2411c1b2c26ede3009b02a67ca88f6e85f55f701e854 not found: ID does not exist" containerID="6307c888d187ca8e02aa2411c1b2c26ede3009b02a67ca88f6e85f55f701e854"
Oct 30 21:55:03.155623 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:03.155588    4412 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6307c888d187ca8e02aa2411c1b2c26ede3009b02a67ca88f6e85f55f701e854"} err="failed to get container status \"6307c888d187ca8e02aa2411c1b2c26ede3009b02a67ca88f6e85f55f701e854\": rpc error: code = NotFound desc = could not find container \"6307c888d187ca8e02aa2411c1b2c26ede3009b02a67ca88f6e85f55f701e854\": container with ID starting with 6307c888d187ca8e02aa2411c1b2c26ede3009b02a67ca88f6e85f55f701e854 not found: ID does not exist"
Oct 30 21:55:03.155623 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:03.155598    4412 scope.go:122] "RemoveContainer" containerID="cf3a5ed14c2e2e1af74992abbc7c89f830f6fd0d2da428bb55bdeae2bb456ba5"
Oct 30 21:55:03.155822 ip-10-0-131-202 kubenswrapper[4412]: E1030 21:55:03.155809    4412 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cf3a5ed14c2e2e1af74992abbc7c89f830f6fd0d2da428bb55bdeae2bb456ba5\": container with ID starting with cf3a5ed14c2e2e1af74992abbc7c89f830f6fd0d2da428bb55bdeae2bb456ba5 not found: ID does not exist" containerID="cf3a5ed14c2e2e1af74992abbc7c89f830f6fd0d2da428bb55bdeae2bb456ba5"
Oct 30 21:55:03.155860 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:03.155825    4412 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cf3a5ed14c2e2e1af74992abbc7c89f830f6fd0d2da428bb55bdeae2bb456ba5"} err="failed to get container status \"cf3a5ed14c2e2e1af74992abbc7c89f830f6fd0d2da428bb55bdeae2bb456ba5\": rpc error: code = NotFound desc = could not find container \"cf3a5ed14c2e2e1af74992abbc7c89f830f6fd0d2da428bb55bdeae2bb456ba5\": container with ID starting with cf3a5ed14c2e2e1af74992abbc7c89f830f6fd0d2da428bb55bdeae2bb456ba5 not found: ID does not exist"
Oct 30 21:55:03.271753 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:03.271721    4412 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Oct 30 21:55:03.274821 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:03.274801    4412 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Oct 30 21:55:03.322306 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:03.322285    4412 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Oct 30 21:55:03.322583 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:03.322571    4412 cpu_manager.go:414] "containerMap: removing container" podUID="92120790-479e-4c07-9572-034b9988f73f" containerName="config-reloader"
Oct 30 21:55:03.322628 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:03.322584    4412 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="92120790-479e-4c07-9572-034b9988f73f" containerName="config-reloader"
Oct 30 21:55:03.322628 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:03.322593    4412 cpu_manager.go:414] "containerMap: removing container" podUID="92120790-479e-4c07-9572-034b9988f73f" containerName="kube-rbac-proxy"
Oct 30 21:55:03.322628 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:03.322597    4412 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="92120790-479e-4c07-9572-034b9988f73f" containerName="kube-rbac-proxy"
Oct 30 21:55:03.322628 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:03.322603    4412 cpu_manager.go:414] "containerMap: removing container" podUID="92120790-479e-4c07-9572-034b9988f73f" containerName="kube-rbac-proxy-metric"
Oct 30 21:55:03.322628 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:03.322608    4412 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="92120790-479e-4c07-9572-034b9988f73f" containerName="kube-rbac-proxy-metric"
Oct 30 21:55:03.322628 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:03.322623    4412 cpu_manager.go:414] "containerMap: removing container" podUID="92120790-479e-4c07-9572-034b9988f73f" containerName="alertmanager"
Oct 30 21:55:03.322628 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:03.322628    4412 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="92120790-479e-4c07-9572-034b9988f73f" containerName="alertmanager"
Oct 30 21:55:03.322848 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:03.322634    4412 cpu_manager.go:414] "containerMap: removing container" podUID="92120790-479e-4c07-9572-034b9988f73f" containerName="kube-rbac-proxy-web"
Oct 30 21:55:03.322848 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:03.322639    4412 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="92120790-479e-4c07-9572-034b9988f73f" containerName="kube-rbac-proxy-web"
Oct 30 21:55:03.322848 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:03.322652    4412 cpu_manager.go:414] "containerMap: removing container" podUID="92120790-479e-4c07-9572-034b9988f73f" containerName="prom-label-proxy"
Oct 30 21:55:03.322848 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:03.322657    4412 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="92120790-479e-4c07-9572-034b9988f73f" containerName="prom-label-proxy"
Oct 30 21:55:03.322848 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:03.322662    4412 cpu_manager.go:414] "containerMap: removing container" podUID="92120790-479e-4c07-9572-034b9988f73f" containerName="init-config-reloader"
Oct 30 21:55:03.322848 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:03.322667    4412 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="92120790-479e-4c07-9572-034b9988f73f" containerName="init-config-reloader"
Oct 30 21:55:03.322848 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:03.322707    4412 memory_manager.go:365] "RemoveStaleState removing state" podUID="92120790-479e-4c07-9572-034b9988f73f" containerName="prom-label-proxy"
Oct 30 21:55:03.322848 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:03.322715    4412 memory_manager.go:365] "RemoveStaleState removing state" podUID="92120790-479e-4c07-9572-034b9988f73f" containerName="config-reloader"
Oct 30 21:55:03.322848 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:03.322720    4412 memory_manager.go:365] "RemoveStaleState removing state" podUID="92120790-479e-4c07-9572-034b9988f73f" containerName="kube-rbac-proxy"
Oct 30 21:55:03.322848 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:03.322726    4412 memory_manager.go:365] "RemoveStaleState removing state" podUID="92120790-479e-4c07-9572-034b9988f73f" containerName="kube-rbac-proxy-metric"
Oct 30 21:55:03.322848 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:03.322733    4412 memory_manager.go:365] "RemoveStaleState removing state" podUID="92120790-479e-4c07-9572-034b9988f73f" containerName="alertmanager"
Oct 30 21:55:03.322848 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:03.322739    4412 memory_manager.go:365] "RemoveStaleState removing state" podUID="92120790-479e-4c07-9572-034b9988f73f" containerName="kube-rbac-proxy-web"
Oct 30 21:55:03.327844 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:03.327829    4412 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:55:03.381498 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:03.381436    4412 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-psvrw\" (UniqueName: \"kubernetes.io/projected/02d2dde4-4ed7-4b5a-b639-f54c97ba473c-kube-api-access-psvrw\") pod \"alertmanager-main-0\" (UID: \"02d2dde4-4ed7-4b5a-b639-f54c97ba473c\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:55:03.381498 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:03.381465    4412 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/02d2dde4-4ed7-4b5a-b639-f54c97ba473c-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"02d2dde4-4ed7-4b5a-b639-f54c97ba473c\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:55:03.381498 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:03.381492    4412 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/02d2dde4-4ed7-4b5a-b639-f54c97ba473c-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"02d2dde4-4ed7-4b5a-b639-f54c97ba473c\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:55:03.381725 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:03.381696    4412 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/02d2dde4-4ed7-4b5a-b639-f54c97ba473c-config-volume\") pod \"alertmanager-main-0\" (UID: \"02d2dde4-4ed7-4b5a-b639-f54c97ba473c\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:55:03.381771 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:03.381730    4412 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/02d2dde4-4ed7-4b5a-b639-f54c97ba473c-config-out\") pod \"alertmanager-main-0\" (UID: \"02d2dde4-4ed7-4b5a-b639-f54c97ba473c\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:55:03.381870 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:03.381802    4412 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/02d2dde4-4ed7-4b5a-b639-f54c97ba473c-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"02d2dde4-4ed7-4b5a-b639-f54c97ba473c\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:55:03.381870 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:03.381842    4412 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/02d2dde4-4ed7-4b5a-b639-f54c97ba473c-web-config\") pod \"alertmanager-main-0\" (UID: \"02d2dde4-4ed7-4b5a-b639-f54c97ba473c\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:55:03.381974 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:03.381886    4412 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/02d2dde4-4ed7-4b5a-b639-f54c97ba473c-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"02d2dde4-4ed7-4b5a-b639-f54c97ba473c\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:55:03.381974 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:03.381942    4412 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/02d2dde4-4ed7-4b5a-b639-f54c97ba473c-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"02d2dde4-4ed7-4b5a-b639-f54c97ba473c\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:55:03.381974 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:03.381966    4412 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/02d2dde4-4ed7-4b5a-b639-f54c97ba473c-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"02d2dde4-4ed7-4b5a-b639-f54c97ba473c\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:55:03.382157 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:03.382048    4412 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/02d2dde4-4ed7-4b5a-b639-f54c97ba473c-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"02d2dde4-4ed7-4b5a-b639-f54c97ba473c\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:55:03.382157 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:03.382105    4412 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/02d2dde4-4ed7-4b5a-b639-f54c97ba473c-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"02d2dde4-4ed7-4b5a-b639-f54c97ba473c\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:55:03.382157 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:03.382151    4412 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/02d2dde4-4ed7-4b5a-b639-f54c97ba473c-tls-assets\") pod \"alertmanager-main-0\" (UID: \"02d2dde4-4ed7-4b5a-b639-f54c97ba473c\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:55:03.382440 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:03.382423    4412 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Oct 30 21:55:03.401345 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:03.401325    4412 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Oct 30 21:55:03.401776 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:03.401755    4412 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Oct 30 21:55:03.401874 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:03.401785    4412 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Oct 30 21:55:03.401874 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:03.401784    4412 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Oct 30 21:55:03.401874 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:03.401805    4412 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Oct 30 21:55:03.431938 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:03.431917    4412 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-rps27\""
Oct 30 21:55:03.431938 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:03.431930    4412 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Oct 30 21:55:03.431938 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:03.431924    4412 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Oct 30 21:55:03.432098 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:03.431942    4412 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Oct 30 21:55:03.443302 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:03.443274    4412 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Oct 30 21:55:03.483312 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:03.483287    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/02d2dde4-4ed7-4b5a-b639-f54c97ba473c-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"02d2dde4-4ed7-4b5a-b639-f54c97ba473c\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:55:03.483435 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:03.483325    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/02d2dde4-4ed7-4b5a-b639-f54c97ba473c-config-volume\") pod \"alertmanager-main-0\" (UID: \"02d2dde4-4ed7-4b5a-b639-f54c97ba473c\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:55:03.483435 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:03.483344    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/02d2dde4-4ed7-4b5a-b639-f54c97ba473c-config-out\") pod \"alertmanager-main-0\" (UID: \"02d2dde4-4ed7-4b5a-b639-f54c97ba473c\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:55:03.483505 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:03.483456    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/02d2dde4-4ed7-4b5a-b639-f54c97ba473c-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"02d2dde4-4ed7-4b5a-b639-f54c97ba473c\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:55:03.483505 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:03.483494    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/02d2dde4-4ed7-4b5a-b639-f54c97ba473c-web-config\") pod \"alertmanager-main-0\" (UID: \"02d2dde4-4ed7-4b5a-b639-f54c97ba473c\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:55:03.483580 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:03.483520    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/02d2dde4-4ed7-4b5a-b639-f54c97ba473c-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"02d2dde4-4ed7-4b5a-b639-f54c97ba473c\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:55:03.483630 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:03.483574    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/02d2dde4-4ed7-4b5a-b639-f54c97ba473c-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"02d2dde4-4ed7-4b5a-b639-f54c97ba473c\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:55:03.483630 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:03.483600    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/02d2dde4-4ed7-4b5a-b639-f54c97ba473c-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"02d2dde4-4ed7-4b5a-b639-f54c97ba473c\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:55:03.483729 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:03.483640    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/02d2dde4-4ed7-4b5a-b639-f54c97ba473c-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"02d2dde4-4ed7-4b5a-b639-f54c97ba473c\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:55:03.483729 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:03.483683    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/02d2dde4-4ed7-4b5a-b639-f54c97ba473c-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"02d2dde4-4ed7-4b5a-b639-f54c97ba473c\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:55:03.483729 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:03.483709    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/02d2dde4-4ed7-4b5a-b639-f54c97ba473c-tls-assets\") pod \"alertmanager-main-0\" (UID: \"02d2dde4-4ed7-4b5a-b639-f54c97ba473c\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:55:03.483875 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:03.483741    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-psvrw\" (UniqueName: \"kubernetes.io/projected/02d2dde4-4ed7-4b5a-b639-f54c97ba473c-kube-api-access-psvrw\") pod \"alertmanager-main-0\" (UID: \"02d2dde4-4ed7-4b5a-b639-f54c97ba473c\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:55:03.483875 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:03.483771    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/02d2dde4-4ed7-4b5a-b639-f54c97ba473c-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"02d2dde4-4ed7-4b5a-b639-f54c97ba473c\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:55:03.484623 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:03.484597    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/02d2dde4-4ed7-4b5a-b639-f54c97ba473c-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"02d2dde4-4ed7-4b5a-b639-f54c97ba473c\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:55:03.485278 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:03.484720    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/02d2dde4-4ed7-4b5a-b639-f54c97ba473c-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"02d2dde4-4ed7-4b5a-b639-f54c97ba473c\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:55:03.485415 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:03.485307    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/02d2dde4-4ed7-4b5a-b639-f54c97ba473c-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"02d2dde4-4ed7-4b5a-b639-f54c97ba473c\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:55:03.486629 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:03.486596    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/02d2dde4-4ed7-4b5a-b639-f54c97ba473c-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"02d2dde4-4ed7-4b5a-b639-f54c97ba473c\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:55:03.486762 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:03.486733    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/02d2dde4-4ed7-4b5a-b639-f54c97ba473c-web-config\") pod \"alertmanager-main-0\" (UID: \"02d2dde4-4ed7-4b5a-b639-f54c97ba473c\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:55:03.487261 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:03.487215    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/02d2dde4-4ed7-4b5a-b639-f54c97ba473c-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"02d2dde4-4ed7-4b5a-b639-f54c97ba473c\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:55:03.487261 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:03.487247    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/02d2dde4-4ed7-4b5a-b639-f54c97ba473c-config-out\") pod \"alertmanager-main-0\" (UID: \"02d2dde4-4ed7-4b5a-b639-f54c97ba473c\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:55:03.487370 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:03.487284    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/02d2dde4-4ed7-4b5a-b639-f54c97ba473c-config-volume\") pod \"alertmanager-main-0\" (UID: \"02d2dde4-4ed7-4b5a-b639-f54c97ba473c\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:55:03.487488 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:03.487465    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/02d2dde4-4ed7-4b5a-b639-f54c97ba473c-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"02d2dde4-4ed7-4b5a-b639-f54c97ba473c\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:55:03.487659 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:03.487636    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/02d2dde4-4ed7-4b5a-b639-f54c97ba473c-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"02d2dde4-4ed7-4b5a-b639-f54c97ba473c\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:55:03.488410 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:03.488385    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/02d2dde4-4ed7-4b5a-b639-f54c97ba473c-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"02d2dde4-4ed7-4b5a-b639-f54c97ba473c\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:55:03.488678 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:03.488656    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/02d2dde4-4ed7-4b5a-b639-f54c97ba473c-tls-assets\") pod \"alertmanager-main-0\" (UID: \"02d2dde4-4ed7-4b5a-b639-f54c97ba473c\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:55:03.542254 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:03.542231    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-psvrw\" (UniqueName: \"kubernetes.io/projected/02d2dde4-4ed7-4b5a-b639-f54c97ba473c-kube-api-access-psvrw\") pod \"alertmanager-main-0\" (UID: \"02d2dde4-4ed7-4b5a-b639-f54c97ba473c\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:55:03.625644 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:03.625616    4412 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="92120790-479e-4c07-9572-034b9988f73f" path="/var/lib/kubelet/pods/92120790-479e-4c07-9572-034b9988f73f/volumes"
Oct 30 21:55:03.652090 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:03.652033    4412 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:55:04.082163 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:04.082134    4412 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Oct 30 21:55:04.084999 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:55:04.084973    4412 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod02d2dde4_4ed7_4b5a_b639_f54c97ba473c.slice/crio-d174ac06ab00537d2adabd655f60efb0f50696ee6a431d0f2c49dd6633a01337 WatchSource:0}: Error finding container d174ac06ab00537d2adabd655f60efb0f50696ee6a431d0f2c49dd6633a01337: Status 404 returned error can't find the container with id d174ac06ab00537d2adabd655f60efb0f50696ee6a431d0f2c49dd6633a01337
Oct 30 21:55:04.107506 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:04.107477    4412 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"02d2dde4-4ed7-4b5a-b639-f54c97ba473c","Type":"ContainerStarted","Data":"d174ac06ab00537d2adabd655f60efb0f50696ee6a431d0f2c49dd6633a01337"}
Oct 30 21:55:04.447854 ip-10-0-131-202 kubenswrapper[4412]: E1030 21:55:04.447774    4412 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bkhmz" containerName="extractor"
Oct 30 21:55:05.112808 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:05.112776    4412 generic.go:356] "Generic (PLEG): container finished" podID="02d2dde4-4ed7-4b5a-b639-f54c97ba473c" containerID="16b91ccaf128e51bdd28bba6c42b1e5da665534c115bd864f060c22c78d060de" exitCode=0
Oct 30 21:55:05.113231 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:05.112850    4412 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"02d2dde4-4ed7-4b5a-b639-f54c97ba473c","Type":"ContainerDied","Data":"16b91ccaf128e51bdd28bba6c42b1e5da665534c115bd864f060c22c78d060de"}
Oct 30 21:55:06.119828 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:06.119791    4412 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"02d2dde4-4ed7-4b5a-b639-f54c97ba473c","Type":"ContainerStarted","Data":"b6b6a4c88cf9304a26fbdd30cd3497996b136d6e372ea485277eea765c2dad36"}
Oct 30 21:55:06.120282 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:06.119859    4412 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"02d2dde4-4ed7-4b5a-b639-f54c97ba473c","Type":"ContainerStarted","Data":"aecfc1c99929b2b8083d2190356bb933e46c553d0bfa7f50b37d565cbd203c0f"}
Oct 30 21:55:06.120282 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:06.119886    4412 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"02d2dde4-4ed7-4b5a-b639-f54c97ba473c","Type":"ContainerStarted","Data":"d30d950a4f40660477014cbe6a11bea85e57e606fb5814924543200b6c0259ac"}
Oct 30 21:55:06.120282 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:06.119921    4412 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"02d2dde4-4ed7-4b5a-b639-f54c97ba473c","Type":"ContainerStarted","Data":"5470f4e5ad8a383c68940d3521a6663cc8d21240d355c3739afa656b0fbc7ee3"}
Oct 30 21:55:06.120282 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:06.119947    4412 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"02d2dde4-4ed7-4b5a-b639-f54c97ba473c","Type":"ContainerStarted","Data":"52b5e29e35438c535da79d541a6a669b9ca4297c0a3a372519d3a1e8b5f00474"}
Oct 30 21:55:06.120282 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:06.119970    4412 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"02d2dde4-4ed7-4b5a-b639-f54c97ba473c","Type":"ContainerStarted","Data":"c9135445a9917aae43934d2fc08ffa959f37577cf8bad13051fe25874a315c7e"}
Oct 30 21:55:06.246906 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:06.246850    4412 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=3.234486195 podStartE2EDuration="3.246836325s" totalImagesPullingTime="0s" totalInitContainerRuntime="12.35013ms" isStatefulPod=false podCreationTimestamp="2026-10-30 21:55:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:55:06.246807824 +0000 UTC m=+126.751479775" watchObservedRunningTime="2026-10-30 21:55:06.246836325 +0000 UTC m=+126.751508276"
Oct 30 21:55:07.032181 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:07.032140    4412 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Oct 30 21:55:07.032481 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:07.032443    4412 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="12507178-e97f-4c11-9a1b-1f398101d1ed" containerName="prometheus" containerID="cri-o://20160ccf3bff21dff5b3b07c02829aada5e4b2b6b75d8715957176258b975205" gracePeriod=600
Oct 30 21:55:07.032481 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:07.032469    4412 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="12507178-e97f-4c11-9a1b-1f398101d1ed" containerName="kube-rbac-proxy" containerID="cri-o://8751397d963b2f8b29d7a15e3df56dd34a68504efb93929a396fa224b48dc3fe" gracePeriod=600
Oct 30 21:55:07.032717 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:07.032505    4412 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="12507178-e97f-4c11-9a1b-1f398101d1ed" containerName="kube-rbac-proxy-web" containerID="cri-o://5cab152d119b329ca208021d7239b647136dcf716754c8ac35e9f5f54b8d49d1" gracePeriod=600
Oct 30 21:55:07.032717 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:07.032493    4412 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="12507178-e97f-4c11-9a1b-1f398101d1ed" containerName="config-reloader" containerID="cri-o://583cf6620b731785d813dc08e3347f393468afe31c01671a5d09b8cd7d35fdfa" gracePeriod=600
Oct 30 21:55:07.032717 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:07.032541    4412 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="12507178-e97f-4c11-9a1b-1f398101d1ed" containerName="kube-rbac-proxy-thanos" containerID="cri-o://7d47b973d606a4331f6014c56255bbf9594fd1d3ddf41e42e9b1c176bae99d26" gracePeriod=600
Oct 30 21:55:07.032717 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:07.032477    4412 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="12507178-e97f-4c11-9a1b-1f398101d1ed" containerName="thanos-sidecar" containerID="cri-o://1a1379b25dc909eb29ef51c8fadb581d667bbde25e26c978f2d42691f1c4eee0" gracePeriod=600
Oct 30 21:55:07.058277 ip-10-0-131-202 kubenswrapper[4412]: E1030 21:55:07.057799    4412 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Canceled desc = context canceled" containerID="20160ccf3bff21dff5b3b07c02829aada5e4b2b6b75d8715957176258b975205" cmd=["sh","-c","if [ -x \"$(command -v curl)\" ]; then exec curl --fail http://localhost:9090/-/ready; elif [ -x \"$(command -v wget)\" ]; then exec wget -q -O /dev/null http://localhost:9090/-/ready; else exit 1; fi"]
Oct 30 21:55:07.058277 ip-10-0-131-202 kubenswrapper[4412]: E1030 21:55:07.057866    4412 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Canceled desc = context canceled" containerID="20160ccf3bff21dff5b3b07c02829aada5e4b2b6b75d8715957176258b975205" cmd=["sh","-c","if [ -x \"$(command -v curl)\" ]; then exec curl --fail http://localhost:9090/-/ready; elif [ -x \"$(command -v wget)\" ]; then exec wget -q -O /dev/null http://localhost:9090/-/ready; else exit 1; fi"]
Oct 30 21:55:07.058277 ip-10-0-131-202 kubenswrapper[4412]: E1030 21:55:07.057893    4412 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Canceled desc = context canceled" containerID="20160ccf3bff21dff5b3b07c02829aada5e4b2b6b75d8715957176258b975205" cmd=["sh","-c","if [ -x \"$(command -v curl)\" ]; then exec curl --fail http://localhost:9090/-/ready; elif [ -x \"$(command -v wget)\" ]; then exec wget -q -O /dev/null http://localhost:9090/-/ready; else exit 1; fi"]
Oct 30 21:55:07.058277 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:07.057916    4412 prober.go:106] "Probe errored" probeType="Readiness" pod="openshift-monitoring/prometheus-k8s-0" podUID="12507178-e97f-4c11-9a1b-1f398101d1ed" containerName="prometheus" probeResult="unknown" err="rpc error: code = Canceled desc = context canceled"
Oct 30 21:55:07.286148 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:07.286080    4412 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:55:07.416692 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:07.416664    4412 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/12507178-e97f-4c11-9a1b-1f398101d1ed-secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/12507178-e97f-4c11-9a1b-1f398101d1ed-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"12507178-e97f-4c11-9a1b-1f398101d1ed\" (UID: \"12507178-e97f-4c11-9a1b-1f398101d1ed\") "
Oct 30 21:55:07.416816 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:07.416700    4412 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/12507178-e97f-4c11-9a1b-1f398101d1ed-secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/12507178-e97f-4c11-9a1b-1f398101d1ed-secret-kube-rbac-proxy\") pod \"12507178-e97f-4c11-9a1b-1f398101d1ed\" (UID: \"12507178-e97f-4c11-9a1b-1f398101d1ed\") "
Oct 30 21:55:07.416816 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:07.416724    4412 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/12507178-e97f-4c11-9a1b-1f398101d1ed-prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/12507178-e97f-4c11-9a1b-1f398101d1ed-prometheus-k8s-rulefiles-0\") pod \"12507178-e97f-4c11-9a1b-1f398101d1ed\" (UID: \"12507178-e97f-4c11-9a1b-1f398101d1ed\") "
Oct 30 21:55:07.416816 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:07.416750    4412 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/12507178-e97f-4c11-9a1b-1f398101d1ed-secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/12507178-e97f-4c11-9a1b-1f398101d1ed-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"12507178-e97f-4c11-9a1b-1f398101d1ed\" (UID: \"12507178-e97f-4c11-9a1b-1f398101d1ed\") "
Oct 30 21:55:07.416816 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:07.416775    4412 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/12507178-e97f-4c11-9a1b-1f398101d1ed-configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/12507178-e97f-4c11-9a1b-1f398101d1ed-configmap-metrics-client-ca\") pod \"12507178-e97f-4c11-9a1b-1f398101d1ed\" (UID: \"12507178-e97f-4c11-9a1b-1f398101d1ed\") "
Oct 30 21:55:07.416816 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:07.416799    4412 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/12507178-e97f-4c11-9a1b-1f398101d1ed-configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/12507178-e97f-4c11-9a1b-1f398101d1ed-configmap-serving-certs-ca-bundle\") pod \"12507178-e97f-4c11-9a1b-1f398101d1ed\" (UID: \"12507178-e97f-4c11-9a1b-1f398101d1ed\") "
Oct 30 21:55:07.417063 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:07.416818    4412 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/12507178-e97f-4c11-9a1b-1f398101d1ed-prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/12507178-e97f-4c11-9a1b-1f398101d1ed-prometheus-trusted-ca-bundle\") pod \"12507178-e97f-4c11-9a1b-1f398101d1ed\" (UID: \"12507178-e97f-4c11-9a1b-1f398101d1ed\") "
Oct 30 21:55:07.417063 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:07.416847    4412 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/12507178-e97f-4c11-9a1b-1f398101d1ed-prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/12507178-e97f-4c11-9a1b-1f398101d1ed-prometheus-k8s-rulefiles-2\") pod \"12507178-e97f-4c11-9a1b-1f398101d1ed\" (UID: \"12507178-e97f-4c11-9a1b-1f398101d1ed\") "
Oct 30 21:55:07.417063 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:07.416877    4412 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/12507178-e97f-4c11-9a1b-1f398101d1ed-config-out\" (UniqueName: \"kubernetes.io/empty-dir/12507178-e97f-4c11-9a1b-1f398101d1ed-config-out\") pod \"12507178-e97f-4c11-9a1b-1f398101d1ed\" (UID: \"12507178-e97f-4c11-9a1b-1f398101d1ed\") "
Oct 30 21:55:07.417063 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:07.416918    4412 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/12507178-e97f-4c11-9a1b-1f398101d1ed-kube-api-access-mn7ls\" (UniqueName: \"kubernetes.io/projected/12507178-e97f-4c11-9a1b-1f398101d1ed-kube-api-access-mn7ls\") pod \"12507178-e97f-4c11-9a1b-1f398101d1ed\" (UID: \"12507178-e97f-4c11-9a1b-1f398101d1ed\") "
Oct 30 21:55:07.417063 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:07.416950    4412 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/12507178-e97f-4c11-9a1b-1f398101d1ed-configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/12507178-e97f-4c11-9a1b-1f398101d1ed-configmap-kubelet-serving-ca-bundle\") pod \"12507178-e97f-4c11-9a1b-1f398101d1ed\" (UID: \"12507178-e97f-4c11-9a1b-1f398101d1ed\") "
Oct 30 21:55:07.417063 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:07.416976    4412 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/12507178-e97f-4c11-9a1b-1f398101d1ed-config\" (UniqueName: \"kubernetes.io/secret/12507178-e97f-4c11-9a1b-1f398101d1ed-config\") pod \"12507178-e97f-4c11-9a1b-1f398101d1ed\" (UID: \"12507178-e97f-4c11-9a1b-1f398101d1ed\") "
Oct 30 21:55:07.417063 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:07.417002    4412 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/12507178-e97f-4c11-9a1b-1f398101d1ed-tls-assets\" (UniqueName: \"kubernetes.io/projected/12507178-e97f-4c11-9a1b-1f398101d1ed-tls-assets\") pod \"12507178-e97f-4c11-9a1b-1f398101d1ed\" (UID: \"12507178-e97f-4c11-9a1b-1f398101d1ed\") "
Oct 30 21:55:07.417063 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:07.417026    4412 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/12507178-e97f-4c11-9a1b-1f398101d1ed-secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/12507178-e97f-4c11-9a1b-1f398101d1ed-secret-grpc-tls\") pod \"12507178-e97f-4c11-9a1b-1f398101d1ed\" (UID: \"12507178-e97f-4c11-9a1b-1f398101d1ed\") "
Oct 30 21:55:07.417063 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:07.417054    4412 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/12507178-e97f-4c11-9a1b-1f398101d1ed-prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/12507178-e97f-4c11-9a1b-1f398101d1ed-prometheus-k8s-db\") pod \"12507178-e97f-4c11-9a1b-1f398101d1ed\" (UID: \"12507178-e97f-4c11-9a1b-1f398101d1ed\") "
Oct 30 21:55:07.417540 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:07.417100    4412 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/12507178-e97f-4c11-9a1b-1f398101d1ed-thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/12507178-e97f-4c11-9a1b-1f398101d1ed-thanos-prometheus-http-client-file\") pod \"12507178-e97f-4c11-9a1b-1f398101d1ed\" (UID: \"12507178-e97f-4c11-9a1b-1f398101d1ed\") "
Oct 30 21:55:07.417540 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:07.417155    4412 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/12507178-e97f-4c11-9a1b-1f398101d1ed-web-config\" (UniqueName: \"kubernetes.io/secret/12507178-e97f-4c11-9a1b-1f398101d1ed-web-config\") pod \"12507178-e97f-4c11-9a1b-1f398101d1ed\" (UID: \"12507178-e97f-4c11-9a1b-1f398101d1ed\") "
Oct 30 21:55:07.417540 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:07.417196    4412 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/12507178-e97f-4c11-9a1b-1f398101d1ed-secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/12507178-e97f-4c11-9a1b-1f398101d1ed-secret-metrics-client-certs\") pod \"12507178-e97f-4c11-9a1b-1f398101d1ed\" (UID: \"12507178-e97f-4c11-9a1b-1f398101d1ed\") "
Oct 30 21:55:07.417540 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:07.417232    4412 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/12507178-e97f-4c11-9a1b-1f398101d1ed-secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/12507178-e97f-4c11-9a1b-1f398101d1ed-secret-prometheus-k8s-tls\") pod \"12507178-e97f-4c11-9a1b-1f398101d1ed\" (UID: \"12507178-e97f-4c11-9a1b-1f398101d1ed\") "
Oct 30 21:55:07.417540 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:07.417267    4412 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/12507178-e97f-4c11-9a1b-1f398101d1ed-prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/12507178-e97f-4c11-9a1b-1f398101d1ed-prometheus-k8s-rulefiles-1\") pod \"12507178-e97f-4c11-9a1b-1f398101d1ed\" (UID: \"12507178-e97f-4c11-9a1b-1f398101d1ed\") "
Oct 30 21:55:07.417540 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:07.417275    4412 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/12507178-e97f-4c11-9a1b-1f398101d1ed-configmap-metrics-client-ca" pod "12507178-e97f-4c11-9a1b-1f398101d1ed" (UID: "12507178-e97f-4c11-9a1b-1f398101d1ed"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:55:07.417540 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:07.417335    4412 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/12507178-e97f-4c11-9a1b-1f398101d1ed-configmap-serving-certs-ca-bundle" pod "12507178-e97f-4c11-9a1b-1f398101d1ed" (UID: "12507178-e97f-4c11-9a1b-1f398101d1ed"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:55:07.417853 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:07.417615    4412 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/12507178-e97f-4c11-9a1b-1f398101d1ed-configmap-metrics-client-ca\") on node \"ip-10-0-131-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:55:07.417853 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:07.417636    4412 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/12507178-e97f-4c11-9a1b-1f398101d1ed-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-131-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:55:07.417853 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:07.417617    4412 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/12507178-e97f-4c11-9a1b-1f398101d1ed-prometheus-k8s-rulefiles-1" pod "12507178-e97f-4c11-9a1b-1f398101d1ed" (UID: "12507178-e97f-4c11-9a1b-1f398101d1ed"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:55:07.417853 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:07.417696    4412 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/12507178-e97f-4c11-9a1b-1f398101d1ed-prometheus-k8s-rulefiles-2" pod "12507178-e97f-4c11-9a1b-1f398101d1ed" (UID: "12507178-e97f-4c11-9a1b-1f398101d1ed"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:55:07.418422 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:07.418053    4412 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/12507178-e97f-4c11-9a1b-1f398101d1ed-configmap-kubelet-serving-ca-bundle" pod "12507178-e97f-4c11-9a1b-1f398101d1ed" (UID: "12507178-e97f-4c11-9a1b-1f398101d1ed"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:55:07.418422 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:07.418238    4412 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/12507178-e97f-4c11-9a1b-1f398101d1ed-prometheus-trusted-ca-bundle" pod "12507178-e97f-4c11-9a1b-1f398101d1ed" (UID: "12507178-e97f-4c11-9a1b-1f398101d1ed"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:55:07.418827 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:07.418751    4412 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/12507178-e97f-4c11-9a1b-1f398101d1ed-prometheus-k8s-rulefiles-0" pod "12507178-e97f-4c11-9a1b-1f398101d1ed" (UID: "12507178-e97f-4c11-9a1b-1f398101d1ed"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:55:07.419208 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:07.419170    4412 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/12507178-e97f-4c11-9a1b-1f398101d1ed-prometheus-k8s-db" pod "12507178-e97f-4c11-9a1b-1f398101d1ed" (UID: "12507178-e97f-4c11-9a1b-1f398101d1ed"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:55:07.420531 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:07.420500    4412 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/12507178-e97f-4c11-9a1b-1f398101d1ed-secret-prometheus-k8s-kube-rbac-proxy-web" pod "12507178-e97f-4c11-9a1b-1f398101d1ed" (UID: "12507178-e97f-4c11-9a1b-1f398101d1ed"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:55:07.420621 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:07.420543    4412 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/12507178-e97f-4c11-9a1b-1f398101d1ed-secret-prometheus-k8s-thanos-sidecar-tls" pod "12507178-e97f-4c11-9a1b-1f398101d1ed" (UID: "12507178-e97f-4c11-9a1b-1f398101d1ed"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:55:07.421097 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:07.421062    4412 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/12507178-e97f-4c11-9a1b-1f398101d1ed-secret-metrics-client-certs" pod "12507178-e97f-4c11-9a1b-1f398101d1ed" (UID: "12507178-e97f-4c11-9a1b-1f398101d1ed"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:55:07.421222 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:07.421108    4412 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/12507178-e97f-4c11-9a1b-1f398101d1ed-secret-prometheus-k8s-tls" pod "12507178-e97f-4c11-9a1b-1f398101d1ed" (UID: "12507178-e97f-4c11-9a1b-1f398101d1ed"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:55:07.421288 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:07.421261    4412 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/12507178-e97f-4c11-9a1b-1f398101d1ed-tls-assets" pod "12507178-e97f-4c11-9a1b-1f398101d1ed" (UID: "12507178-e97f-4c11-9a1b-1f398101d1ed"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:55:07.421581 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:07.421560    4412 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/12507178-e97f-4c11-9a1b-1f398101d1ed-secret-kube-rbac-proxy" pod "12507178-e97f-4c11-9a1b-1f398101d1ed" (UID: "12507178-e97f-4c11-9a1b-1f398101d1ed"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:55:07.421847 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:07.421819    4412 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/12507178-e97f-4c11-9a1b-1f398101d1ed-kube-api-access-mn7ls" pod "12507178-e97f-4c11-9a1b-1f398101d1ed" (UID: "12507178-e97f-4c11-9a1b-1f398101d1ed"). InnerVolumeSpecName "kube-api-access-mn7ls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:55:07.422103 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:07.422086    4412 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/12507178-e97f-4c11-9a1b-1f398101d1ed-secret-grpc-tls" pod "12507178-e97f-4c11-9a1b-1f398101d1ed" (UID: "12507178-e97f-4c11-9a1b-1f398101d1ed"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:55:07.422306 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:07.422290    4412 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/12507178-e97f-4c11-9a1b-1f398101d1ed-config" pod "12507178-e97f-4c11-9a1b-1f398101d1ed" (UID: "12507178-e97f-4c11-9a1b-1f398101d1ed"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:55:07.422360 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:07.422342    4412 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/12507178-e97f-4c11-9a1b-1f398101d1ed-thanos-prometheus-http-client-file" pod "12507178-e97f-4c11-9a1b-1f398101d1ed" (UID: "12507178-e97f-4c11-9a1b-1f398101d1ed"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:55:07.422900 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:07.422885    4412 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/12507178-e97f-4c11-9a1b-1f398101d1ed-config-out" pod "12507178-e97f-4c11-9a1b-1f398101d1ed" (UID: "12507178-e97f-4c11-9a1b-1f398101d1ed"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:55:07.431583 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:07.431563    4412 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/12507178-e97f-4c11-9a1b-1f398101d1ed-web-config" pod "12507178-e97f-4c11-9a1b-1f398101d1ed" (UID: "12507178-e97f-4c11-9a1b-1f398101d1ed"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:55:07.518732 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:07.518704    4412 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/12507178-e97f-4c11-9a1b-1f398101d1ed-secret-grpc-tls\") on node \"ip-10-0-131-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:55:07.518732 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:07.518728    4412 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/12507178-e97f-4c11-9a1b-1f398101d1ed-prometheus-k8s-db\") on node \"ip-10-0-131-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:55:07.518732 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:07.518738    4412 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/12507178-e97f-4c11-9a1b-1f398101d1ed-thanos-prometheus-http-client-file\") on node \"ip-10-0-131-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:55:07.518926 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:07.518748    4412 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/12507178-e97f-4c11-9a1b-1f398101d1ed-web-config\") on node \"ip-10-0-131-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:55:07.518926 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:07.518758    4412 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/12507178-e97f-4c11-9a1b-1f398101d1ed-secret-metrics-client-certs\") on node \"ip-10-0-131-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:55:07.518926 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:07.518767    4412 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/12507178-e97f-4c11-9a1b-1f398101d1ed-secret-prometheus-k8s-tls\") on node \"ip-10-0-131-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:55:07.518926 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:07.518776    4412 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/12507178-e97f-4c11-9a1b-1f398101d1ed-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-131-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:55:07.518926 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:07.518790    4412 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/12507178-e97f-4c11-9a1b-1f398101d1ed-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-131-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:55:07.518926 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:07.518809    4412 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/12507178-e97f-4c11-9a1b-1f398101d1ed-secret-kube-rbac-proxy\") on node \"ip-10-0-131-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:55:07.518926 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:07.518818    4412 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/12507178-e97f-4c11-9a1b-1f398101d1ed-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-131-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:55:07.518926 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:07.518826    4412 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/12507178-e97f-4c11-9a1b-1f398101d1ed-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-131-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:55:07.518926 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:07.518835    4412 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/12507178-e97f-4c11-9a1b-1f398101d1ed-prometheus-trusted-ca-bundle\") on node \"ip-10-0-131-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:55:07.518926 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:07.518843    4412 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/12507178-e97f-4c11-9a1b-1f398101d1ed-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-131-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:55:07.518926 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:07.518851    4412 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/12507178-e97f-4c11-9a1b-1f398101d1ed-config-out\") on node \"ip-10-0-131-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:55:07.518926 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:07.518873    4412 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mn7ls\" (UniqueName: \"kubernetes.io/projected/12507178-e97f-4c11-9a1b-1f398101d1ed-kube-api-access-mn7ls\") on node \"ip-10-0-131-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:55:07.518926 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:07.518881    4412 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/12507178-e97f-4c11-9a1b-1f398101d1ed-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-131-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:55:07.518926 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:07.518891    4412 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/12507178-e97f-4c11-9a1b-1f398101d1ed-config\") on node \"ip-10-0-131-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:55:07.518926 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:07.518899    4412 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/12507178-e97f-4c11-9a1b-1f398101d1ed-tls-assets\") on node \"ip-10-0-131-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 21:55:08.132318 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:08.132285    4412 generic.go:356] "Generic (PLEG): container finished" podID="12507178-e97f-4c11-9a1b-1f398101d1ed" containerID="7d47b973d606a4331f6014c56255bbf9594fd1d3ddf41e42e9b1c176bae99d26" exitCode=0
Oct 30 21:55:08.132318 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:08.132311    4412 generic.go:356] "Generic (PLEG): container finished" podID="12507178-e97f-4c11-9a1b-1f398101d1ed" containerID="8751397d963b2f8b29d7a15e3df56dd34a68504efb93929a396fa224b48dc3fe" exitCode=0
Oct 30 21:55:08.132318 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:08.132319    4412 generic.go:356] "Generic (PLEG): container finished" podID="12507178-e97f-4c11-9a1b-1f398101d1ed" containerID="5cab152d119b329ca208021d7239b647136dcf716754c8ac35e9f5f54b8d49d1" exitCode=0
Oct 30 21:55:08.132318 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:08.132327    4412 generic.go:356] "Generic (PLEG): container finished" podID="12507178-e97f-4c11-9a1b-1f398101d1ed" containerID="1a1379b25dc909eb29ef51c8fadb581d667bbde25e26c978f2d42691f1c4eee0" exitCode=0
Oct 30 21:55:08.132574 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:08.132332    4412 generic.go:356] "Generic (PLEG): container finished" podID="12507178-e97f-4c11-9a1b-1f398101d1ed" containerID="583cf6620b731785d813dc08e3347f393468afe31c01671a5d09b8cd7d35fdfa" exitCode=0
Oct 30 21:55:08.132574 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:08.132337    4412 generic.go:356] "Generic (PLEG): container finished" podID="12507178-e97f-4c11-9a1b-1f398101d1ed" containerID="20160ccf3bff21dff5b3b07c02829aada5e4b2b6b75d8715957176258b975205" exitCode=0
Oct 30 21:55:08.132574 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:08.132375    4412 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:55:08.132574 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:08.132372    4412 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"12507178-e97f-4c11-9a1b-1f398101d1ed","Type":"ContainerDied","Data":"7d47b973d606a4331f6014c56255bbf9594fd1d3ddf41e42e9b1c176bae99d26"}
Oct 30 21:55:08.132574 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:08.132467    4412 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"12507178-e97f-4c11-9a1b-1f398101d1ed","Type":"ContainerDied","Data":"8751397d963b2f8b29d7a15e3df56dd34a68504efb93929a396fa224b48dc3fe"}
Oct 30 21:55:08.132574 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:08.132497    4412 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"12507178-e97f-4c11-9a1b-1f398101d1ed","Type":"ContainerDied","Data":"5cab152d119b329ca208021d7239b647136dcf716754c8ac35e9f5f54b8d49d1"}
Oct 30 21:55:08.132574 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:08.132521    4412 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"12507178-e97f-4c11-9a1b-1f398101d1ed","Type":"ContainerDied","Data":"1a1379b25dc909eb29ef51c8fadb581d667bbde25e26c978f2d42691f1c4eee0"}
Oct 30 21:55:08.132574 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:08.132531    4412 scope.go:122] "RemoveContainer" containerID="7d47b973d606a4331f6014c56255bbf9594fd1d3ddf41e42e9b1c176bae99d26"
Oct 30 21:55:08.132574 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:08.132545    4412 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"12507178-e97f-4c11-9a1b-1f398101d1ed","Type":"ContainerDied","Data":"583cf6620b731785d813dc08e3347f393468afe31c01671a5d09b8cd7d35fdfa"}
Oct 30 21:55:08.132574 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:08.132568    4412 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"12507178-e97f-4c11-9a1b-1f398101d1ed","Type":"ContainerDied","Data":"20160ccf3bff21dff5b3b07c02829aada5e4b2b6b75d8715957176258b975205"}
Oct 30 21:55:08.132958 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:08.132592    4412 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"12507178-e97f-4c11-9a1b-1f398101d1ed","Type":"ContainerDied","Data":"3102a387134ebbcddc6a1f857287a0069a6b8eadf4f3eb4343aac61b5d152e1e"}
Oct 30 21:55:08.139841 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:08.139824    4412 scope.go:122] "RemoveContainer" containerID="8751397d963b2f8b29d7a15e3df56dd34a68504efb93929a396fa224b48dc3fe"
Oct 30 21:55:08.146544 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:08.146530    4412 scope.go:122] "RemoveContainer" containerID="5cab152d119b329ca208021d7239b647136dcf716754c8ac35e9f5f54b8d49d1"
Oct 30 21:55:08.152791 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:08.152748    4412 scope.go:122] "RemoveContainer" containerID="1a1379b25dc909eb29ef51c8fadb581d667bbde25e26c978f2d42691f1c4eee0"
Oct 30 21:55:08.159053 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:08.159038    4412 scope.go:122] "RemoveContainer" containerID="583cf6620b731785d813dc08e3347f393468afe31c01671a5d09b8cd7d35fdfa"
Oct 30 21:55:08.165329 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:08.165311    4412 scope.go:122] "RemoveContainer" containerID="20160ccf3bff21dff5b3b07c02829aada5e4b2b6b75d8715957176258b975205"
Oct 30 21:55:08.172340 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:08.172324    4412 scope.go:122] "RemoveContainer" containerID="cfb715bdde296b49e62e8d3dc1680612dad490050b54bce29720b044592a9054"
Oct 30 21:55:08.178468 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:08.178453    4412 scope.go:122] "RemoveContainer" containerID="7d47b973d606a4331f6014c56255bbf9594fd1d3ddf41e42e9b1c176bae99d26"
Oct 30 21:55:08.178708 ip-10-0-131-202 kubenswrapper[4412]: E1030 21:55:08.178689    4412 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7d47b973d606a4331f6014c56255bbf9594fd1d3ddf41e42e9b1c176bae99d26\": container with ID starting with 7d47b973d606a4331f6014c56255bbf9594fd1d3ddf41e42e9b1c176bae99d26 not found: ID does not exist" containerID="7d47b973d606a4331f6014c56255bbf9594fd1d3ddf41e42e9b1c176bae99d26"
Oct 30 21:55:08.178758 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:08.178717    4412 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7d47b973d606a4331f6014c56255bbf9594fd1d3ddf41e42e9b1c176bae99d26"} err="failed to get container status \"7d47b973d606a4331f6014c56255bbf9594fd1d3ddf41e42e9b1c176bae99d26\": rpc error: code = NotFound desc = could not find container \"7d47b973d606a4331f6014c56255bbf9594fd1d3ddf41e42e9b1c176bae99d26\": container with ID starting with 7d47b973d606a4331f6014c56255bbf9594fd1d3ddf41e42e9b1c176bae99d26 not found: ID does not exist"
Oct 30 21:55:08.178758 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:08.178736    4412 scope.go:122] "RemoveContainer" containerID="8751397d963b2f8b29d7a15e3df56dd34a68504efb93929a396fa224b48dc3fe"
Oct 30 21:55:08.178978 ip-10-0-131-202 kubenswrapper[4412]: E1030 21:55:08.178961    4412 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8751397d963b2f8b29d7a15e3df56dd34a68504efb93929a396fa224b48dc3fe\": container with ID starting with 8751397d963b2f8b29d7a15e3df56dd34a68504efb93929a396fa224b48dc3fe not found: ID does not exist" containerID="8751397d963b2f8b29d7a15e3df56dd34a68504efb93929a396fa224b48dc3fe"
Oct 30 21:55:08.179045 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:08.178984    4412 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8751397d963b2f8b29d7a15e3df56dd34a68504efb93929a396fa224b48dc3fe"} err="failed to get container status \"8751397d963b2f8b29d7a15e3df56dd34a68504efb93929a396fa224b48dc3fe\": rpc error: code = NotFound desc = could not find container \"8751397d963b2f8b29d7a15e3df56dd34a68504efb93929a396fa224b48dc3fe\": container with ID starting with 8751397d963b2f8b29d7a15e3df56dd34a68504efb93929a396fa224b48dc3fe not found: ID does not exist"
Oct 30 21:55:08.179045 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:08.178999    4412 scope.go:122] "RemoveContainer" containerID="5cab152d119b329ca208021d7239b647136dcf716754c8ac35e9f5f54b8d49d1"
Oct 30 21:55:08.179258 ip-10-0-131-202 kubenswrapper[4412]: E1030 21:55:08.179242    4412 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5cab152d119b329ca208021d7239b647136dcf716754c8ac35e9f5f54b8d49d1\": container with ID starting with 5cab152d119b329ca208021d7239b647136dcf716754c8ac35e9f5f54b8d49d1 not found: ID does not exist" containerID="5cab152d119b329ca208021d7239b647136dcf716754c8ac35e9f5f54b8d49d1"
Oct 30 21:55:08.179300 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:08.179263    4412 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5cab152d119b329ca208021d7239b647136dcf716754c8ac35e9f5f54b8d49d1"} err="failed to get container status \"5cab152d119b329ca208021d7239b647136dcf716754c8ac35e9f5f54b8d49d1\": rpc error: code = NotFound desc = could not find container \"5cab152d119b329ca208021d7239b647136dcf716754c8ac35e9f5f54b8d49d1\": container with ID starting with 5cab152d119b329ca208021d7239b647136dcf716754c8ac35e9f5f54b8d49d1 not found: ID does not exist"
Oct 30 21:55:08.179300 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:08.179277    4412 scope.go:122] "RemoveContainer" containerID="1a1379b25dc909eb29ef51c8fadb581d667bbde25e26c978f2d42691f1c4eee0"
Oct 30 21:55:08.179475 ip-10-0-131-202 kubenswrapper[4412]: E1030 21:55:08.179460    4412 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1a1379b25dc909eb29ef51c8fadb581d667bbde25e26c978f2d42691f1c4eee0\": container with ID starting with 1a1379b25dc909eb29ef51c8fadb581d667bbde25e26c978f2d42691f1c4eee0 not found: ID does not exist" containerID="1a1379b25dc909eb29ef51c8fadb581d667bbde25e26c978f2d42691f1c4eee0"
Oct 30 21:55:08.179520 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:08.179477    4412 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1a1379b25dc909eb29ef51c8fadb581d667bbde25e26c978f2d42691f1c4eee0"} err="failed to get container status \"1a1379b25dc909eb29ef51c8fadb581d667bbde25e26c978f2d42691f1c4eee0\": rpc error: code = NotFound desc = could not find container \"1a1379b25dc909eb29ef51c8fadb581d667bbde25e26c978f2d42691f1c4eee0\": container with ID starting with 1a1379b25dc909eb29ef51c8fadb581d667bbde25e26c978f2d42691f1c4eee0 not found: ID does not exist"
Oct 30 21:55:08.179520 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:08.179489    4412 scope.go:122] "RemoveContainer" containerID="583cf6620b731785d813dc08e3347f393468afe31c01671a5d09b8cd7d35fdfa"
Oct 30 21:55:08.179687 ip-10-0-131-202 kubenswrapper[4412]: E1030 21:55:08.179672    4412 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"583cf6620b731785d813dc08e3347f393468afe31c01671a5d09b8cd7d35fdfa\": container with ID starting with 583cf6620b731785d813dc08e3347f393468afe31c01671a5d09b8cd7d35fdfa not found: ID does not exist" containerID="583cf6620b731785d813dc08e3347f393468afe31c01671a5d09b8cd7d35fdfa"
Oct 30 21:55:08.179726 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:08.179690    4412 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"583cf6620b731785d813dc08e3347f393468afe31c01671a5d09b8cd7d35fdfa"} err="failed to get container status \"583cf6620b731785d813dc08e3347f393468afe31c01671a5d09b8cd7d35fdfa\": rpc error: code = NotFound desc = could not find container \"583cf6620b731785d813dc08e3347f393468afe31c01671a5d09b8cd7d35fdfa\": container with ID starting with 583cf6620b731785d813dc08e3347f393468afe31c01671a5d09b8cd7d35fdfa not found: ID does not exist"
Oct 30 21:55:08.179726 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:08.179701    4412 scope.go:122] "RemoveContainer" containerID="20160ccf3bff21dff5b3b07c02829aada5e4b2b6b75d8715957176258b975205"
Oct 30 21:55:08.179924 ip-10-0-131-202 kubenswrapper[4412]: E1030 21:55:08.179909    4412 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"20160ccf3bff21dff5b3b07c02829aada5e4b2b6b75d8715957176258b975205\": container with ID starting with 20160ccf3bff21dff5b3b07c02829aada5e4b2b6b75d8715957176258b975205 not found: ID does not exist" containerID="20160ccf3bff21dff5b3b07c02829aada5e4b2b6b75d8715957176258b975205"
Oct 30 21:55:08.179962 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:08.179927    4412 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"20160ccf3bff21dff5b3b07c02829aada5e4b2b6b75d8715957176258b975205"} err="failed to get container status \"20160ccf3bff21dff5b3b07c02829aada5e4b2b6b75d8715957176258b975205\": rpc error: code = NotFound desc = could not find container \"20160ccf3bff21dff5b3b07c02829aada5e4b2b6b75d8715957176258b975205\": container with ID starting with 20160ccf3bff21dff5b3b07c02829aada5e4b2b6b75d8715957176258b975205 not found: ID does not exist"
Oct 30 21:55:08.179962 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:08.179938    4412 scope.go:122] "RemoveContainer" containerID="cfb715bdde296b49e62e8d3dc1680612dad490050b54bce29720b044592a9054"
Oct 30 21:55:08.180168 ip-10-0-131-202 kubenswrapper[4412]: E1030 21:55:08.180151    4412 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cfb715bdde296b49e62e8d3dc1680612dad490050b54bce29720b044592a9054\": container with ID starting with cfb715bdde296b49e62e8d3dc1680612dad490050b54bce29720b044592a9054 not found: ID does not exist" containerID="cfb715bdde296b49e62e8d3dc1680612dad490050b54bce29720b044592a9054"
Oct 30 21:55:08.180228 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:08.180176    4412 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cfb715bdde296b49e62e8d3dc1680612dad490050b54bce29720b044592a9054"} err="failed to get container status \"cfb715bdde296b49e62e8d3dc1680612dad490050b54bce29720b044592a9054\": rpc error: code = NotFound desc = could not find container \"cfb715bdde296b49e62e8d3dc1680612dad490050b54bce29720b044592a9054\": container with ID starting with cfb715bdde296b49e62e8d3dc1680612dad490050b54bce29720b044592a9054 not found: ID does not exist"
Oct 30 21:55:08.180228 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:08.180195    4412 scope.go:122] "RemoveContainer" containerID="7d47b973d606a4331f6014c56255bbf9594fd1d3ddf41e42e9b1c176bae99d26"
Oct 30 21:55:08.180424 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:08.180408    4412 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7d47b973d606a4331f6014c56255bbf9594fd1d3ddf41e42e9b1c176bae99d26"} err="failed to get container status \"7d47b973d606a4331f6014c56255bbf9594fd1d3ddf41e42e9b1c176bae99d26\": rpc error: code = NotFound desc = could not find container \"7d47b973d606a4331f6014c56255bbf9594fd1d3ddf41e42e9b1c176bae99d26\": container with ID starting with 7d47b973d606a4331f6014c56255bbf9594fd1d3ddf41e42e9b1c176bae99d26 not found: ID does not exist"
Oct 30 21:55:08.180485 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:08.180425    4412 scope.go:122] "RemoveContainer" containerID="8751397d963b2f8b29d7a15e3df56dd34a68504efb93929a396fa224b48dc3fe"
Oct 30 21:55:08.180641 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:08.180622    4412 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8751397d963b2f8b29d7a15e3df56dd34a68504efb93929a396fa224b48dc3fe"} err="failed to get container status \"8751397d963b2f8b29d7a15e3df56dd34a68504efb93929a396fa224b48dc3fe\": rpc error: code = NotFound desc = could not find container \"8751397d963b2f8b29d7a15e3df56dd34a68504efb93929a396fa224b48dc3fe\": container with ID starting with 8751397d963b2f8b29d7a15e3df56dd34a68504efb93929a396fa224b48dc3fe not found: ID does not exist"
Oct 30 21:55:08.180641 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:08.180639    4412 scope.go:122] "RemoveContainer" containerID="5cab152d119b329ca208021d7239b647136dcf716754c8ac35e9f5f54b8d49d1"
Oct 30 21:55:08.180836 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:08.180816    4412 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5cab152d119b329ca208021d7239b647136dcf716754c8ac35e9f5f54b8d49d1"} err="failed to get container status \"5cab152d119b329ca208021d7239b647136dcf716754c8ac35e9f5f54b8d49d1\": rpc error: code = NotFound desc = could not find container \"5cab152d119b329ca208021d7239b647136dcf716754c8ac35e9f5f54b8d49d1\": container with ID starting with 5cab152d119b329ca208021d7239b647136dcf716754c8ac35e9f5f54b8d49d1 not found: ID does not exist"
Oct 30 21:55:08.180912 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:08.180838    4412 scope.go:122] "RemoveContainer" containerID="1a1379b25dc909eb29ef51c8fadb581d667bbde25e26c978f2d42691f1c4eee0"
Oct 30 21:55:08.181054 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:08.181038    4412 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1a1379b25dc909eb29ef51c8fadb581d667bbde25e26c978f2d42691f1c4eee0"} err="failed to get container status \"1a1379b25dc909eb29ef51c8fadb581d667bbde25e26c978f2d42691f1c4eee0\": rpc error: code = NotFound desc = could not find container \"1a1379b25dc909eb29ef51c8fadb581d667bbde25e26c978f2d42691f1c4eee0\": container with ID starting with 1a1379b25dc909eb29ef51c8fadb581d667bbde25e26c978f2d42691f1c4eee0 not found: ID does not exist"
Oct 30 21:55:08.181098 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:08.181055    4412 scope.go:122] "RemoveContainer" containerID="583cf6620b731785d813dc08e3347f393468afe31c01671a5d09b8cd7d35fdfa"
Oct 30 21:55:08.181281 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:08.181264    4412 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"583cf6620b731785d813dc08e3347f393468afe31c01671a5d09b8cd7d35fdfa"} err="failed to get container status \"583cf6620b731785d813dc08e3347f393468afe31c01671a5d09b8cd7d35fdfa\": rpc error: code = NotFound desc = could not find container \"583cf6620b731785d813dc08e3347f393468afe31c01671a5d09b8cd7d35fdfa\": container with ID starting with 583cf6620b731785d813dc08e3347f393468afe31c01671a5d09b8cd7d35fdfa not found: ID does not exist"
Oct 30 21:55:08.181333 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:08.181281    4412 scope.go:122] "RemoveContainer" containerID="20160ccf3bff21dff5b3b07c02829aada5e4b2b6b75d8715957176258b975205"
Oct 30 21:55:08.181494 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:08.181473    4412 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"20160ccf3bff21dff5b3b07c02829aada5e4b2b6b75d8715957176258b975205"} err="failed to get container status \"20160ccf3bff21dff5b3b07c02829aada5e4b2b6b75d8715957176258b975205\": rpc error: code = NotFound desc = could not find container \"20160ccf3bff21dff5b3b07c02829aada5e4b2b6b75d8715957176258b975205\": container with ID starting with 20160ccf3bff21dff5b3b07c02829aada5e4b2b6b75d8715957176258b975205 not found: ID does not exist"
Oct 30 21:55:08.181544 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:08.181494    4412 scope.go:122] "RemoveContainer" containerID="cfb715bdde296b49e62e8d3dc1680612dad490050b54bce29720b044592a9054"
Oct 30 21:55:08.181696 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:08.181680    4412 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cfb715bdde296b49e62e8d3dc1680612dad490050b54bce29720b044592a9054"} err="failed to get container status \"cfb715bdde296b49e62e8d3dc1680612dad490050b54bce29720b044592a9054\": rpc error: code = NotFound desc = could not find container \"cfb715bdde296b49e62e8d3dc1680612dad490050b54bce29720b044592a9054\": container with ID starting with cfb715bdde296b49e62e8d3dc1680612dad490050b54bce29720b044592a9054 not found: ID does not exist"
Oct 30 21:55:08.181696 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:08.181695    4412 scope.go:122] "RemoveContainer" containerID="7d47b973d606a4331f6014c56255bbf9594fd1d3ddf41e42e9b1c176bae99d26"
Oct 30 21:55:08.181887 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:08.181873    4412 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7d47b973d606a4331f6014c56255bbf9594fd1d3ddf41e42e9b1c176bae99d26"} err="failed to get container status \"7d47b973d606a4331f6014c56255bbf9594fd1d3ddf41e42e9b1c176bae99d26\": rpc error: code = NotFound desc = could not find container \"7d47b973d606a4331f6014c56255bbf9594fd1d3ddf41e42e9b1c176bae99d26\": container with ID starting with 7d47b973d606a4331f6014c56255bbf9594fd1d3ddf41e42e9b1c176bae99d26 not found: ID does not exist"
Oct 30 21:55:08.181934 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:08.181887    4412 scope.go:122] "RemoveContainer" containerID="8751397d963b2f8b29d7a15e3df56dd34a68504efb93929a396fa224b48dc3fe"
Oct 30 21:55:08.182076 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:08.182062    4412 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8751397d963b2f8b29d7a15e3df56dd34a68504efb93929a396fa224b48dc3fe"} err="failed to get container status \"8751397d963b2f8b29d7a15e3df56dd34a68504efb93929a396fa224b48dc3fe\": rpc error: code = NotFound desc = could not find container \"8751397d963b2f8b29d7a15e3df56dd34a68504efb93929a396fa224b48dc3fe\": container with ID starting with 8751397d963b2f8b29d7a15e3df56dd34a68504efb93929a396fa224b48dc3fe not found: ID does not exist"
Oct 30 21:55:08.182147 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:08.182076    4412 scope.go:122] "RemoveContainer" containerID="5cab152d119b329ca208021d7239b647136dcf716754c8ac35e9f5f54b8d49d1"
Oct 30 21:55:08.182328 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:08.182309    4412 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5cab152d119b329ca208021d7239b647136dcf716754c8ac35e9f5f54b8d49d1"} err="failed to get container status \"5cab152d119b329ca208021d7239b647136dcf716754c8ac35e9f5f54b8d49d1\": rpc error: code = NotFound desc = could not find container \"5cab152d119b329ca208021d7239b647136dcf716754c8ac35e9f5f54b8d49d1\": container with ID starting with 5cab152d119b329ca208021d7239b647136dcf716754c8ac35e9f5f54b8d49d1 not found: ID does not exist"
Oct 30 21:55:08.182386 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:08.182329    4412 scope.go:122] "RemoveContainer" containerID="1a1379b25dc909eb29ef51c8fadb581d667bbde25e26c978f2d42691f1c4eee0"
Oct 30 21:55:08.182515 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:08.182500    4412 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1a1379b25dc909eb29ef51c8fadb581d667bbde25e26c978f2d42691f1c4eee0"} err="failed to get container status \"1a1379b25dc909eb29ef51c8fadb581d667bbde25e26c978f2d42691f1c4eee0\": rpc error: code = NotFound desc = could not find container \"1a1379b25dc909eb29ef51c8fadb581d667bbde25e26c978f2d42691f1c4eee0\": container with ID starting with 1a1379b25dc909eb29ef51c8fadb581d667bbde25e26c978f2d42691f1c4eee0 not found: ID does not exist"
Oct 30 21:55:08.182564 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:08.182515    4412 scope.go:122] "RemoveContainer" containerID="583cf6620b731785d813dc08e3347f393468afe31c01671a5d09b8cd7d35fdfa"
Oct 30 21:55:08.182690 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:08.182675    4412 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"583cf6620b731785d813dc08e3347f393468afe31c01671a5d09b8cd7d35fdfa"} err="failed to get container status \"583cf6620b731785d813dc08e3347f393468afe31c01671a5d09b8cd7d35fdfa\": rpc error: code = NotFound desc = could not find container \"583cf6620b731785d813dc08e3347f393468afe31c01671a5d09b8cd7d35fdfa\": container with ID starting with 583cf6620b731785d813dc08e3347f393468afe31c01671a5d09b8cd7d35fdfa not found: ID does not exist"
Oct 30 21:55:08.182690 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:08.182688    4412 scope.go:122] "RemoveContainer" containerID="20160ccf3bff21dff5b3b07c02829aada5e4b2b6b75d8715957176258b975205"
Oct 30 21:55:08.182846 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:08.182833    4412 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"20160ccf3bff21dff5b3b07c02829aada5e4b2b6b75d8715957176258b975205"} err="failed to get container status \"20160ccf3bff21dff5b3b07c02829aada5e4b2b6b75d8715957176258b975205\": rpc error: code = NotFound desc = could not find container \"20160ccf3bff21dff5b3b07c02829aada5e4b2b6b75d8715957176258b975205\": container with ID starting with 20160ccf3bff21dff5b3b07c02829aada5e4b2b6b75d8715957176258b975205 not found: ID does not exist"
Oct 30 21:55:08.182899 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:08.182846    4412 scope.go:122] "RemoveContainer" containerID="cfb715bdde296b49e62e8d3dc1680612dad490050b54bce29720b044592a9054"
Oct 30 21:55:08.183032 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:08.183016    4412 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cfb715bdde296b49e62e8d3dc1680612dad490050b54bce29720b044592a9054"} err="failed to get container status \"cfb715bdde296b49e62e8d3dc1680612dad490050b54bce29720b044592a9054\": rpc error: code = NotFound desc = could not find container \"cfb715bdde296b49e62e8d3dc1680612dad490050b54bce29720b044592a9054\": container with ID starting with cfb715bdde296b49e62e8d3dc1680612dad490050b54bce29720b044592a9054 not found: ID does not exist"
Oct 30 21:55:08.183032 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:08.183031    4412 scope.go:122] "RemoveContainer" containerID="7d47b973d606a4331f6014c56255bbf9594fd1d3ddf41e42e9b1c176bae99d26"
Oct 30 21:55:08.183222 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:08.183202    4412 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7d47b973d606a4331f6014c56255bbf9594fd1d3ddf41e42e9b1c176bae99d26"} err="failed to get container status \"7d47b973d606a4331f6014c56255bbf9594fd1d3ddf41e42e9b1c176bae99d26\": rpc error: code = NotFound desc = could not find container \"7d47b973d606a4331f6014c56255bbf9594fd1d3ddf41e42e9b1c176bae99d26\": container with ID starting with 7d47b973d606a4331f6014c56255bbf9594fd1d3ddf41e42e9b1c176bae99d26 not found: ID does not exist"
Oct 30 21:55:08.183260 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:08.183224    4412 scope.go:122] "RemoveContainer" containerID="8751397d963b2f8b29d7a15e3df56dd34a68504efb93929a396fa224b48dc3fe"
Oct 30 21:55:08.183415 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:08.183401    4412 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8751397d963b2f8b29d7a15e3df56dd34a68504efb93929a396fa224b48dc3fe"} err="failed to get container status \"8751397d963b2f8b29d7a15e3df56dd34a68504efb93929a396fa224b48dc3fe\": rpc error: code = NotFound desc = could not find container \"8751397d963b2f8b29d7a15e3df56dd34a68504efb93929a396fa224b48dc3fe\": container with ID starting with 8751397d963b2f8b29d7a15e3df56dd34a68504efb93929a396fa224b48dc3fe not found: ID does not exist"
Oct 30 21:55:08.183454 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:08.183414    4412 scope.go:122] "RemoveContainer" containerID="5cab152d119b329ca208021d7239b647136dcf716754c8ac35e9f5f54b8d49d1"
Oct 30 21:55:08.183601 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:08.183584    4412 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5cab152d119b329ca208021d7239b647136dcf716754c8ac35e9f5f54b8d49d1"} err="failed to get container status \"5cab152d119b329ca208021d7239b647136dcf716754c8ac35e9f5f54b8d49d1\": rpc error: code = NotFound desc = could not find container \"5cab152d119b329ca208021d7239b647136dcf716754c8ac35e9f5f54b8d49d1\": container with ID starting with 5cab152d119b329ca208021d7239b647136dcf716754c8ac35e9f5f54b8d49d1 not found: ID does not exist"
Oct 30 21:55:08.183643 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:08.183603    4412 scope.go:122] "RemoveContainer" containerID="1a1379b25dc909eb29ef51c8fadb581d667bbde25e26c978f2d42691f1c4eee0"
Oct 30 21:55:08.183818 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:08.183801    4412 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1a1379b25dc909eb29ef51c8fadb581d667bbde25e26c978f2d42691f1c4eee0"} err="failed to get container status \"1a1379b25dc909eb29ef51c8fadb581d667bbde25e26c978f2d42691f1c4eee0\": rpc error: code = NotFound desc = could not find container \"1a1379b25dc909eb29ef51c8fadb581d667bbde25e26c978f2d42691f1c4eee0\": container with ID starting with 1a1379b25dc909eb29ef51c8fadb581d667bbde25e26c978f2d42691f1c4eee0 not found: ID does not exist"
Oct 30 21:55:08.183818 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:08.183814    4412 scope.go:122] "RemoveContainer" containerID="583cf6620b731785d813dc08e3347f393468afe31c01671a5d09b8cd7d35fdfa"
Oct 30 21:55:08.184030 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:08.184017    4412 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"583cf6620b731785d813dc08e3347f393468afe31c01671a5d09b8cd7d35fdfa"} err="failed to get container status \"583cf6620b731785d813dc08e3347f393468afe31c01671a5d09b8cd7d35fdfa\": rpc error: code = NotFound desc = could not find container \"583cf6620b731785d813dc08e3347f393468afe31c01671a5d09b8cd7d35fdfa\": container with ID starting with 583cf6620b731785d813dc08e3347f393468afe31c01671a5d09b8cd7d35fdfa not found: ID does not exist"
Oct 30 21:55:08.184030 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:08.184030    4412 scope.go:122] "RemoveContainer" containerID="20160ccf3bff21dff5b3b07c02829aada5e4b2b6b75d8715957176258b975205"
Oct 30 21:55:08.184269 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:08.184252    4412 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"20160ccf3bff21dff5b3b07c02829aada5e4b2b6b75d8715957176258b975205"} err="failed to get container status \"20160ccf3bff21dff5b3b07c02829aada5e4b2b6b75d8715957176258b975205\": rpc error: code = NotFound desc = could not find container \"20160ccf3bff21dff5b3b07c02829aada5e4b2b6b75d8715957176258b975205\": container with ID starting with 20160ccf3bff21dff5b3b07c02829aada5e4b2b6b75d8715957176258b975205 not found: ID does not exist"
Oct 30 21:55:08.184269 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:08.184269    4412 scope.go:122] "RemoveContainer" containerID="cfb715bdde296b49e62e8d3dc1680612dad490050b54bce29720b044592a9054"
Oct 30 21:55:08.184480 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:08.184465    4412 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cfb715bdde296b49e62e8d3dc1680612dad490050b54bce29720b044592a9054"} err="failed to get container status \"cfb715bdde296b49e62e8d3dc1680612dad490050b54bce29720b044592a9054\": rpc error: code = NotFound desc = could not find container \"cfb715bdde296b49e62e8d3dc1680612dad490050b54bce29720b044592a9054\": container with ID starting with cfb715bdde296b49e62e8d3dc1680612dad490050b54bce29720b044592a9054 not found: ID does not exist"
Oct 30 21:55:08.184480 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:08.184478    4412 scope.go:122] "RemoveContainer" containerID="7d47b973d606a4331f6014c56255bbf9594fd1d3ddf41e42e9b1c176bae99d26"
Oct 30 21:55:08.184688 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:08.184668    4412 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7d47b973d606a4331f6014c56255bbf9594fd1d3ddf41e42e9b1c176bae99d26"} err="failed to get container status \"7d47b973d606a4331f6014c56255bbf9594fd1d3ddf41e42e9b1c176bae99d26\": rpc error: code = NotFound desc = could not find container \"7d47b973d606a4331f6014c56255bbf9594fd1d3ddf41e42e9b1c176bae99d26\": container with ID starting with 7d47b973d606a4331f6014c56255bbf9594fd1d3ddf41e42e9b1c176bae99d26 not found: ID does not exist"
Oct 30 21:55:08.184734 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:08.184689    4412 scope.go:122] "RemoveContainer" containerID="8751397d963b2f8b29d7a15e3df56dd34a68504efb93929a396fa224b48dc3fe"
Oct 30 21:55:08.184871 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:08.184856    4412 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8751397d963b2f8b29d7a15e3df56dd34a68504efb93929a396fa224b48dc3fe"} err="failed to get container status \"8751397d963b2f8b29d7a15e3df56dd34a68504efb93929a396fa224b48dc3fe\": rpc error: code = NotFound desc = could not find container \"8751397d963b2f8b29d7a15e3df56dd34a68504efb93929a396fa224b48dc3fe\": container with ID starting with 8751397d963b2f8b29d7a15e3df56dd34a68504efb93929a396fa224b48dc3fe not found: ID does not exist"
Oct 30 21:55:08.184871 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:08.184870    4412 scope.go:122] "RemoveContainer" containerID="5cab152d119b329ca208021d7239b647136dcf716754c8ac35e9f5f54b8d49d1"
Oct 30 21:55:08.185046 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:08.185032    4412 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5cab152d119b329ca208021d7239b647136dcf716754c8ac35e9f5f54b8d49d1"} err="failed to get container status \"5cab152d119b329ca208021d7239b647136dcf716754c8ac35e9f5f54b8d49d1\": rpc error: code = NotFound desc = could not find container \"5cab152d119b329ca208021d7239b647136dcf716754c8ac35e9f5f54b8d49d1\": container with ID starting with 5cab152d119b329ca208021d7239b647136dcf716754c8ac35e9f5f54b8d49d1 not found: ID does not exist"
Oct 30 21:55:08.185085 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:08.185048    4412 scope.go:122] "RemoveContainer" containerID="1a1379b25dc909eb29ef51c8fadb581d667bbde25e26c978f2d42691f1c4eee0"
Oct 30 21:55:08.185245 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:08.185230    4412 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1a1379b25dc909eb29ef51c8fadb581d667bbde25e26c978f2d42691f1c4eee0"} err="failed to get container status \"1a1379b25dc909eb29ef51c8fadb581d667bbde25e26c978f2d42691f1c4eee0\": rpc error: code = NotFound desc = could not find container \"1a1379b25dc909eb29ef51c8fadb581d667bbde25e26c978f2d42691f1c4eee0\": container with ID starting with 1a1379b25dc909eb29ef51c8fadb581d667bbde25e26c978f2d42691f1c4eee0 not found: ID does not exist"
Oct 30 21:55:08.185293 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:08.185245    4412 scope.go:122] "RemoveContainer" containerID="583cf6620b731785d813dc08e3347f393468afe31c01671a5d09b8cd7d35fdfa"
Oct 30 21:55:08.185433 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:08.185415    4412 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"583cf6620b731785d813dc08e3347f393468afe31c01671a5d09b8cd7d35fdfa"} err="failed to get container status \"583cf6620b731785d813dc08e3347f393468afe31c01671a5d09b8cd7d35fdfa\": rpc error: code = NotFound desc = could not find container \"583cf6620b731785d813dc08e3347f393468afe31c01671a5d09b8cd7d35fdfa\": container with ID starting with 583cf6620b731785d813dc08e3347f393468afe31c01671a5d09b8cd7d35fdfa not found: ID does not exist"
Oct 30 21:55:08.185482 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:08.185433    4412 scope.go:122] "RemoveContainer" containerID="20160ccf3bff21dff5b3b07c02829aada5e4b2b6b75d8715957176258b975205"
Oct 30 21:55:08.185645 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:08.185617    4412 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"20160ccf3bff21dff5b3b07c02829aada5e4b2b6b75d8715957176258b975205"} err="failed to get container status \"20160ccf3bff21dff5b3b07c02829aada5e4b2b6b75d8715957176258b975205\": rpc error: code = NotFound desc = could not find container \"20160ccf3bff21dff5b3b07c02829aada5e4b2b6b75d8715957176258b975205\": container with ID starting with 20160ccf3bff21dff5b3b07c02829aada5e4b2b6b75d8715957176258b975205 not found: ID does not exist"
Oct 30 21:55:08.185703 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:08.185646    4412 scope.go:122] "RemoveContainer" containerID="cfb715bdde296b49e62e8d3dc1680612dad490050b54bce29720b044592a9054"
Oct 30 21:55:08.185896 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:08.185880    4412 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cfb715bdde296b49e62e8d3dc1680612dad490050b54bce29720b044592a9054"} err="failed to get container status \"cfb715bdde296b49e62e8d3dc1680612dad490050b54bce29720b044592a9054\": rpc error: code = NotFound desc = could not find container \"cfb715bdde296b49e62e8d3dc1680612dad490050b54bce29720b044592a9054\": container with ID starting with cfb715bdde296b49e62e8d3dc1680612dad490050b54bce29720b044592a9054 not found: ID does not exist"
Oct 30 21:55:08.185956 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:08.185895    4412 scope.go:122] "RemoveContainer" containerID="7d47b973d606a4331f6014c56255bbf9594fd1d3ddf41e42e9b1c176bae99d26"
Oct 30 21:55:08.186105 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:08.186087    4412 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7d47b973d606a4331f6014c56255bbf9594fd1d3ddf41e42e9b1c176bae99d26"} err="failed to get container status \"7d47b973d606a4331f6014c56255bbf9594fd1d3ddf41e42e9b1c176bae99d26\": rpc error: code = NotFound desc = could not find container \"7d47b973d606a4331f6014c56255bbf9594fd1d3ddf41e42e9b1c176bae99d26\": container with ID starting with 7d47b973d606a4331f6014c56255bbf9594fd1d3ddf41e42e9b1c176bae99d26 not found: ID does not exist"
Oct 30 21:55:08.186210 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:08.186107    4412 scope.go:122] "RemoveContainer" containerID="8751397d963b2f8b29d7a15e3df56dd34a68504efb93929a396fa224b48dc3fe"
Oct 30 21:55:08.186368 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:08.186351    4412 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8751397d963b2f8b29d7a15e3df56dd34a68504efb93929a396fa224b48dc3fe"} err="failed to get container status \"8751397d963b2f8b29d7a15e3df56dd34a68504efb93929a396fa224b48dc3fe\": rpc error: code = NotFound desc = could not find container \"8751397d963b2f8b29d7a15e3df56dd34a68504efb93929a396fa224b48dc3fe\": container with ID starting with 8751397d963b2f8b29d7a15e3df56dd34a68504efb93929a396fa224b48dc3fe not found: ID does not exist"
Oct 30 21:55:08.186408 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:08.186368    4412 scope.go:122] "RemoveContainer" containerID="5cab152d119b329ca208021d7239b647136dcf716754c8ac35e9f5f54b8d49d1"
Oct 30 21:55:08.186594 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:08.186577    4412 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5cab152d119b329ca208021d7239b647136dcf716754c8ac35e9f5f54b8d49d1"} err="failed to get container status \"5cab152d119b329ca208021d7239b647136dcf716754c8ac35e9f5f54b8d49d1\": rpc error: code = NotFound desc = could not find container \"5cab152d119b329ca208021d7239b647136dcf716754c8ac35e9f5f54b8d49d1\": container with ID starting with 5cab152d119b329ca208021d7239b647136dcf716754c8ac35e9f5f54b8d49d1 not found: ID does not exist"
Oct 30 21:55:08.186666 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:08.186596    4412 scope.go:122] "RemoveContainer" containerID="1a1379b25dc909eb29ef51c8fadb581d667bbde25e26c978f2d42691f1c4eee0"
Oct 30 21:55:08.186851 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:08.186834    4412 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1a1379b25dc909eb29ef51c8fadb581d667bbde25e26c978f2d42691f1c4eee0"} err="failed to get container status \"1a1379b25dc909eb29ef51c8fadb581d667bbde25e26c978f2d42691f1c4eee0\": rpc error: code = NotFound desc = could not find container \"1a1379b25dc909eb29ef51c8fadb581d667bbde25e26c978f2d42691f1c4eee0\": container with ID starting with 1a1379b25dc909eb29ef51c8fadb581d667bbde25e26c978f2d42691f1c4eee0 not found: ID does not exist"
Oct 30 21:55:08.186895 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:08.186852    4412 scope.go:122] "RemoveContainer" containerID="583cf6620b731785d813dc08e3347f393468afe31c01671a5d09b8cd7d35fdfa"
Oct 30 21:55:08.187070 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:08.187053    4412 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"583cf6620b731785d813dc08e3347f393468afe31c01671a5d09b8cd7d35fdfa"} err="failed to get container status \"583cf6620b731785d813dc08e3347f393468afe31c01671a5d09b8cd7d35fdfa\": rpc error: code = NotFound desc = could not find container \"583cf6620b731785d813dc08e3347f393468afe31c01671a5d09b8cd7d35fdfa\": container with ID starting with 583cf6620b731785d813dc08e3347f393468afe31c01671a5d09b8cd7d35fdfa not found: ID does not exist"
Oct 30 21:55:08.187154 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:08.187070    4412 scope.go:122] "RemoveContainer" containerID="20160ccf3bff21dff5b3b07c02829aada5e4b2b6b75d8715957176258b975205"
Oct 30 21:55:08.187301 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:08.187283    4412 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"20160ccf3bff21dff5b3b07c02829aada5e4b2b6b75d8715957176258b975205"} err="failed to get container status \"20160ccf3bff21dff5b3b07c02829aada5e4b2b6b75d8715957176258b975205\": rpc error: code = NotFound desc = could not find container \"20160ccf3bff21dff5b3b07c02829aada5e4b2b6b75d8715957176258b975205\": container with ID starting with 20160ccf3bff21dff5b3b07c02829aada5e4b2b6b75d8715957176258b975205 not found: ID does not exist"
Oct 30 21:55:08.187368 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:08.187303    4412 scope.go:122] "RemoveContainer" containerID="cfb715bdde296b49e62e8d3dc1680612dad490050b54bce29720b044592a9054"
Oct 30 21:55:08.187532 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:08.187517    4412 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cfb715bdde296b49e62e8d3dc1680612dad490050b54bce29720b044592a9054"} err="failed to get container status \"cfb715bdde296b49e62e8d3dc1680612dad490050b54bce29720b044592a9054\": rpc error: code = NotFound desc = could not find container \"cfb715bdde296b49e62e8d3dc1680612dad490050b54bce29720b044592a9054\": container with ID starting with cfb715bdde296b49e62e8d3dc1680612dad490050b54bce29720b044592a9054 not found: ID does not exist"
Oct 30 21:55:08.350324 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:08.350246    4412 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Oct 30 21:55:08.400695 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:08.400662    4412 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Oct 30 21:55:08.401364 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:08.401347    4412 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Oct 30 21:55:08.401606 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:08.401594    4412 cpu_manager.go:414] "containerMap: removing container" podUID="12507178-e97f-4c11-9a1b-1f398101d1ed" containerName="init-config-reloader"
Oct 30 21:55:08.401639 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:08.401607    4412 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="12507178-e97f-4c11-9a1b-1f398101d1ed" containerName="init-config-reloader"
Oct 30 21:55:08.401639 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:08.401618    4412 cpu_manager.go:414] "containerMap: removing container" podUID="12507178-e97f-4c11-9a1b-1f398101d1ed" containerName="config-reloader"
Oct 30 21:55:08.401639 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:08.401623    4412 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="12507178-e97f-4c11-9a1b-1f398101d1ed" containerName="config-reloader"
Oct 30 21:55:08.401639 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:08.401629    4412 cpu_manager.go:414] "containerMap: removing container" podUID="12507178-e97f-4c11-9a1b-1f398101d1ed" containerName="kube-rbac-proxy"
Oct 30 21:55:08.401639 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:08.401633    4412 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="12507178-e97f-4c11-9a1b-1f398101d1ed" containerName="kube-rbac-proxy"
Oct 30 21:55:08.401639 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:08.401640    4412 cpu_manager.go:414] "containerMap: removing container" podUID="12507178-e97f-4c11-9a1b-1f398101d1ed" containerName="kube-rbac-proxy-thanos"
Oct 30 21:55:08.401804 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:08.401644    4412 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="12507178-e97f-4c11-9a1b-1f398101d1ed" containerName="kube-rbac-proxy-thanos"
Oct 30 21:55:08.401804 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:08.401654    4412 cpu_manager.go:414] "containerMap: removing container" podUID="12507178-e97f-4c11-9a1b-1f398101d1ed" containerName="kube-rbac-proxy-web"
Oct 30 21:55:08.401804 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:08.401659    4412 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="12507178-e97f-4c11-9a1b-1f398101d1ed" containerName="kube-rbac-proxy-web"
Oct 30 21:55:08.401804 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:08.401670    4412 cpu_manager.go:414] "containerMap: removing container" podUID="12507178-e97f-4c11-9a1b-1f398101d1ed" containerName="prometheus"
Oct 30 21:55:08.401804 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:08.401674    4412 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="12507178-e97f-4c11-9a1b-1f398101d1ed" containerName="prometheus"
Oct 30 21:55:08.401804 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:08.401679    4412 cpu_manager.go:414] "containerMap: removing container" podUID="12507178-e97f-4c11-9a1b-1f398101d1ed" containerName="thanos-sidecar"
Oct 30 21:55:08.401804 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:08.401683    4412 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="12507178-e97f-4c11-9a1b-1f398101d1ed" containerName="thanos-sidecar"
Oct 30 21:55:08.401804 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:08.401723    4412 memory_manager.go:365] "RemoveStaleState removing state" podUID="12507178-e97f-4c11-9a1b-1f398101d1ed" containerName="prometheus"
Oct 30 21:55:08.401804 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:08.401730    4412 memory_manager.go:365] "RemoveStaleState removing state" podUID="12507178-e97f-4c11-9a1b-1f398101d1ed" containerName="thanos-sidecar"
Oct 30 21:55:08.401804 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:08.401739    4412 memory_manager.go:365] "RemoveStaleState removing state" podUID="12507178-e97f-4c11-9a1b-1f398101d1ed" containerName="config-reloader"
Oct 30 21:55:08.401804 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:08.401747    4412 memory_manager.go:365] "RemoveStaleState removing state" podUID="12507178-e97f-4c11-9a1b-1f398101d1ed" containerName="kube-rbac-proxy"
Oct 30 21:55:08.401804 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:08.401752    4412 memory_manager.go:365] "RemoveStaleState removing state" podUID="12507178-e97f-4c11-9a1b-1f398101d1ed" containerName="kube-rbac-proxy-thanos"
Oct 30 21:55:08.401804 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:08.401757    4412 memory_manager.go:365] "RemoveStaleState removing state" podUID="12507178-e97f-4c11-9a1b-1f398101d1ed" containerName="kube-rbac-proxy-web"
Oct 30 21:55:08.406859 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:08.406844    4412 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:55:08.460750 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:08.460721    4412 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Oct 30 21:55:08.460849 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:08.460824    4412 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Oct 30 21:55:08.461035 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:08.460992    4412 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Oct 30 21:55:08.461104 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:08.461059    4412 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Oct 30 21:55:08.461104 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:08.461079    4412 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Oct 30 21:55:08.461266 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:08.461249    4412 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Oct 30 21:55:08.461400 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:08.461336    4412 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Oct 30 21:55:08.461450 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:08.461437    4412 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Oct 30 21:55:08.461497 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:08.461447    4412 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Oct 30 21:55:08.461544 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:08.461495    4412 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Oct 30 21:55:08.461544 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:08.461496    4412 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Oct 30 21:55:08.461640 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:08.461618    4412 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Oct 30 21:55:08.461707 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:08.461691    4412 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Oct 30 21:55:08.461817 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:08.461790    4412 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-f513mko3c0tve\""
Oct 30 21:55:08.461974 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:08.461949    4412 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-jfxm4\""
Oct 30 21:55:08.512503 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:08.512481    4412 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Oct 30 21:55:08.512612 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:08.512596    4412 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Oct 30 21:55:08.526790 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:08.526765    4412 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8kkmp\" (UniqueName: \"kubernetes.io/projected/e2aaf8b0-d53f-4452-92f6-170dd6cb201a-kube-api-access-8kkmp\") pod \"prometheus-k8s-0\" (UID: \"e2aaf8b0-d53f-4452-92f6-170dd6cb201a\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:55:08.526889 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:08.526799    4412 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e2aaf8b0-d53f-4452-92f6-170dd6cb201a-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"e2aaf8b0-d53f-4452-92f6-170dd6cb201a\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:55:08.526889 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:08.526833    4412 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e2aaf8b0-d53f-4452-92f6-170dd6cb201a-config-out\") pod \"prometheus-k8s-0\" (UID: \"e2aaf8b0-d53f-4452-92f6-170dd6cb201a\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:55:08.526984 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:08.526884    4412 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/e2aaf8b0-d53f-4452-92f6-170dd6cb201a-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"e2aaf8b0-d53f-4452-92f6-170dd6cb201a\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:55:08.526984 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:08.526931    4412 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e2aaf8b0-d53f-4452-92f6-170dd6cb201a-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"e2aaf8b0-d53f-4452-92f6-170dd6cb201a\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:55:08.526984 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:08.526966    4412 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/e2aaf8b0-d53f-4452-92f6-170dd6cb201a-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"e2aaf8b0-d53f-4452-92f6-170dd6cb201a\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:55:08.527091 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:08.526999    4412 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e2aaf8b0-d53f-4452-92f6-170dd6cb201a-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e2aaf8b0-d53f-4452-92f6-170dd6cb201a\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:55:08.527091 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:08.527033    4412 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e2aaf8b0-d53f-4452-92f6-170dd6cb201a-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"e2aaf8b0-d53f-4452-92f6-170dd6cb201a\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:55:08.527091 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:08.527086    4412 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/e2aaf8b0-d53f-4452-92f6-170dd6cb201a-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"e2aaf8b0-d53f-4452-92f6-170dd6cb201a\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:55:08.527233 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:08.527110    4412 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/e2aaf8b0-d53f-4452-92f6-170dd6cb201a-config\") pod \"prometheus-k8s-0\" (UID: \"e2aaf8b0-d53f-4452-92f6-170dd6cb201a\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:55:08.527233 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:08.527177    4412 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/e2aaf8b0-d53f-4452-92f6-170dd6cb201a-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"e2aaf8b0-d53f-4452-92f6-170dd6cb201a\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:55:08.527233 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:08.527205    4412 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e2aaf8b0-d53f-4452-92f6-170dd6cb201a-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"e2aaf8b0-d53f-4452-92f6-170dd6cb201a\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:55:08.527233 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:08.527221    4412 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/e2aaf8b0-d53f-4452-92f6-170dd6cb201a-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"e2aaf8b0-d53f-4452-92f6-170dd6cb201a\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:55:08.527420 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:08.527239    4412 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/e2aaf8b0-d53f-4452-92f6-170dd6cb201a-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"e2aaf8b0-d53f-4452-92f6-170dd6cb201a\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:55:08.527420 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:08.527318    4412 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e2aaf8b0-d53f-4452-92f6-170dd6cb201a-web-config\") pod \"prometheus-k8s-0\" (UID: \"e2aaf8b0-d53f-4452-92f6-170dd6cb201a\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:55:08.527420 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:08.527362    4412 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/e2aaf8b0-d53f-4452-92f6-170dd6cb201a-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"e2aaf8b0-d53f-4452-92f6-170dd6cb201a\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:55:08.527420 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:08.527394    4412 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/e2aaf8b0-d53f-4452-92f6-170dd6cb201a-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"e2aaf8b0-d53f-4452-92f6-170dd6cb201a\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:55:08.527595 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:08.527421    4412 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/e2aaf8b0-d53f-4452-92f6-170dd6cb201a-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"e2aaf8b0-d53f-4452-92f6-170dd6cb201a\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:55:08.527595 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:08.527454    4412 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e2aaf8b0-d53f-4452-92f6-170dd6cb201a-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e2aaf8b0-d53f-4452-92f6-170dd6cb201a\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:55:08.527595 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:08.527481    4412 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e2aaf8b0-d53f-4452-92f6-170dd6cb201a-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e2aaf8b0-d53f-4452-92f6-170dd6cb201a\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:55:08.628229 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:08.628173    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/e2aaf8b0-d53f-4452-92f6-170dd6cb201a-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"e2aaf8b0-d53f-4452-92f6-170dd6cb201a\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:55:08.628229 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:08.628205    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/e2aaf8b0-d53f-4452-92f6-170dd6cb201a-config\") pod \"prometheus-k8s-0\" (UID: \"e2aaf8b0-d53f-4452-92f6-170dd6cb201a\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:55:08.628229 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:08.628222    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/e2aaf8b0-d53f-4452-92f6-170dd6cb201a-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"e2aaf8b0-d53f-4452-92f6-170dd6cb201a\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:55:08.628429 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:08.628243    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e2aaf8b0-d53f-4452-92f6-170dd6cb201a-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"e2aaf8b0-d53f-4452-92f6-170dd6cb201a\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:55:08.628429 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:08.628395    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/e2aaf8b0-d53f-4452-92f6-170dd6cb201a-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"e2aaf8b0-d53f-4452-92f6-170dd6cb201a\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:55:08.628533 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:08.628436    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/e2aaf8b0-d53f-4452-92f6-170dd6cb201a-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"e2aaf8b0-d53f-4452-92f6-170dd6cb201a\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:55:08.628533 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:08.628469    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e2aaf8b0-d53f-4452-92f6-170dd6cb201a-web-config\") pod \"prometheus-k8s-0\" (UID: \"e2aaf8b0-d53f-4452-92f6-170dd6cb201a\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:55:08.628533 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:08.628497    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/e2aaf8b0-d53f-4452-92f6-170dd6cb201a-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"e2aaf8b0-d53f-4452-92f6-170dd6cb201a\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:55:08.628533 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:08.628526    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/e2aaf8b0-d53f-4452-92f6-170dd6cb201a-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"e2aaf8b0-d53f-4452-92f6-170dd6cb201a\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:55:08.628730 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:08.628552    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/e2aaf8b0-d53f-4452-92f6-170dd6cb201a-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"e2aaf8b0-d53f-4452-92f6-170dd6cb201a\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:55:08.628730 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:08.628586    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e2aaf8b0-d53f-4452-92f6-170dd6cb201a-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e2aaf8b0-d53f-4452-92f6-170dd6cb201a\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:55:08.628730 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:08.628615    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e2aaf8b0-d53f-4452-92f6-170dd6cb201a-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e2aaf8b0-d53f-4452-92f6-170dd6cb201a\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:55:08.628730 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:08.628647    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8kkmp\" (UniqueName: \"kubernetes.io/projected/e2aaf8b0-d53f-4452-92f6-170dd6cb201a-kube-api-access-8kkmp\") pod \"prometheus-k8s-0\" (UID: \"e2aaf8b0-d53f-4452-92f6-170dd6cb201a\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:55:08.628730 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:08.628675    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e2aaf8b0-d53f-4452-92f6-170dd6cb201a-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"e2aaf8b0-d53f-4452-92f6-170dd6cb201a\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:55:08.628730 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:08.628712    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e2aaf8b0-d53f-4452-92f6-170dd6cb201a-config-out\") pod \"prometheus-k8s-0\" (UID: \"e2aaf8b0-d53f-4452-92f6-170dd6cb201a\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:55:08.629042 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:08.628736    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/e2aaf8b0-d53f-4452-92f6-170dd6cb201a-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"e2aaf8b0-d53f-4452-92f6-170dd6cb201a\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:55:08.629042 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:08.628763    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e2aaf8b0-d53f-4452-92f6-170dd6cb201a-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"e2aaf8b0-d53f-4452-92f6-170dd6cb201a\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:55:08.629042 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:08.628787    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/e2aaf8b0-d53f-4452-92f6-170dd6cb201a-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"e2aaf8b0-d53f-4452-92f6-170dd6cb201a\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:55:08.629042 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:08.628822    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e2aaf8b0-d53f-4452-92f6-170dd6cb201a-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e2aaf8b0-d53f-4452-92f6-170dd6cb201a\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:55:08.629042 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:08.628850    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e2aaf8b0-d53f-4452-92f6-170dd6cb201a-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"e2aaf8b0-d53f-4452-92f6-170dd6cb201a\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:55:08.631742 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:08.631613    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/e2aaf8b0-d53f-4452-92f6-170dd6cb201a-config\") pod \"prometheus-k8s-0\" (UID: \"e2aaf8b0-d53f-4452-92f6-170dd6cb201a\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:55:08.631742 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:08.631731    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/e2aaf8b0-d53f-4452-92f6-170dd6cb201a-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"e2aaf8b0-d53f-4452-92f6-170dd6cb201a\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:55:08.631906 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:08.631782    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/e2aaf8b0-d53f-4452-92f6-170dd6cb201a-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"e2aaf8b0-d53f-4452-92f6-170dd6cb201a\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:55:08.631906 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:08.631782    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e2aaf8b0-d53f-4452-92f6-170dd6cb201a-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"e2aaf8b0-d53f-4452-92f6-170dd6cb201a\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:55:08.632016 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:08.631963    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e2aaf8b0-d53f-4452-92f6-170dd6cb201a-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"e2aaf8b0-d53f-4452-92f6-170dd6cb201a\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:55:08.632161 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:08.632138    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/e2aaf8b0-d53f-4452-92f6-170dd6cb201a-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"e2aaf8b0-d53f-4452-92f6-170dd6cb201a\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:55:08.632488 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:08.632385    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e2aaf8b0-d53f-4452-92f6-170dd6cb201a-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"e2aaf8b0-d53f-4452-92f6-170dd6cb201a\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:55:08.632859 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:08.632596    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/e2aaf8b0-d53f-4452-92f6-170dd6cb201a-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"e2aaf8b0-d53f-4452-92f6-170dd6cb201a\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:55:08.632859 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:08.632613    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/e2aaf8b0-d53f-4452-92f6-170dd6cb201a-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"e2aaf8b0-d53f-4452-92f6-170dd6cb201a\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:55:08.632859 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:08.632663    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e2aaf8b0-d53f-4452-92f6-170dd6cb201a-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e2aaf8b0-d53f-4452-92f6-170dd6cb201a\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:55:08.632859 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:08.632705    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/e2aaf8b0-d53f-4452-92f6-170dd6cb201a-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"e2aaf8b0-d53f-4452-92f6-170dd6cb201a\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:55:08.633900 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:08.633163    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/e2aaf8b0-d53f-4452-92f6-170dd6cb201a-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"e2aaf8b0-d53f-4452-92f6-170dd6cb201a\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:55:08.633900 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:08.633514    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e2aaf8b0-d53f-4452-92f6-170dd6cb201a-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e2aaf8b0-d53f-4452-92f6-170dd6cb201a\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:55:08.633900 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:08.633861    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e2aaf8b0-d53f-4452-92f6-170dd6cb201a-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e2aaf8b0-d53f-4452-92f6-170dd6cb201a\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:55:08.634098 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:08.633958    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/e2aaf8b0-d53f-4452-92f6-170dd6cb201a-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"e2aaf8b0-d53f-4452-92f6-170dd6cb201a\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:55:08.634361 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:08.634340    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e2aaf8b0-d53f-4452-92f6-170dd6cb201a-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"e2aaf8b0-d53f-4452-92f6-170dd6cb201a\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:55:08.634673 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:08.634654    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e2aaf8b0-d53f-4452-92f6-170dd6cb201a-web-config\") pod \"prometheus-k8s-0\" (UID: \"e2aaf8b0-d53f-4452-92f6-170dd6cb201a\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:55:08.635172 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:08.635150    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e2aaf8b0-d53f-4452-92f6-170dd6cb201a-config-out\") pod \"prometheus-k8s-0\" (UID: \"e2aaf8b0-d53f-4452-92f6-170dd6cb201a\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:55:08.635220 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:08.635183    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/e2aaf8b0-d53f-4452-92f6-170dd6cb201a-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"e2aaf8b0-d53f-4452-92f6-170dd6cb201a\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:55:08.640002 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:08.639980    4412 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Oct 30 21:55:08.690869 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:08.690841    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8kkmp\" (UniqueName: \"kubernetes.io/projected/e2aaf8b0-d53f-4452-92f6-170dd6cb201a-kube-api-access-8kkmp\") pod \"prometheus-k8s-0\" (UID: \"e2aaf8b0-d53f-4452-92f6-170dd6cb201a\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:55:08.730258 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:08.730234    4412 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:55:09.166295 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:09.166264    4412 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Oct 30 21:55:09.169063 ip-10-0-131-202 kubenswrapper[4412]: W1030 21:55:09.169037    4412 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode2aaf8b0_d53f_4452_92f6_170dd6cb201a.slice/crio-b6a04f819105c15d8e0611e42194d6893c4537f679c5a919cb2df507d6534af2 WatchSource:0}: Error finding container b6a04f819105c15d8e0611e42194d6893c4537f679c5a919cb2df507d6534af2: Status 404 returned error can't find the container with id b6a04f819105c15d8e0611e42194d6893c4537f679c5a919cb2df507d6534af2
Oct 30 21:55:09.626272 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:09.626245    4412 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="12507178-e97f-4c11-9a1b-1f398101d1ed" path="/var/lib/kubelet/pods/12507178-e97f-4c11-9a1b-1f398101d1ed/volumes"
Oct 30 21:55:10.142764 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:10.142735    4412 generic.go:356] "Generic (PLEG): container finished" podID="e2aaf8b0-d53f-4452-92f6-170dd6cb201a" containerID="cc14777dd90d6441bd83f306fc1b13fae356f67b061ec4ff7661717d05da9f62" exitCode=0
Oct 30 21:55:10.142892 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:10.142798    4412 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e2aaf8b0-d53f-4452-92f6-170dd6cb201a","Type":"ContainerDied","Data":"cc14777dd90d6441bd83f306fc1b13fae356f67b061ec4ff7661717d05da9f62"}
Oct 30 21:55:10.142892 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:10.142879    4412 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e2aaf8b0-d53f-4452-92f6-170dd6cb201a","Type":"ContainerStarted","Data":"b6a04f819105c15d8e0611e42194d6893c4537f679c5a919cb2df507d6534af2"}
Oct 30 21:55:11.153897 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:11.153853    4412 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e2aaf8b0-d53f-4452-92f6-170dd6cb201a","Type":"ContainerStarted","Data":"b8e6782b40eb5d16368b8385c450aea25dd481d22a150d59d2a7c3554b71a2cb"}
Oct 30 21:55:11.154398 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:11.153938    4412 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e2aaf8b0-d53f-4452-92f6-170dd6cb201a","Type":"ContainerStarted","Data":"51e1950ac0e3b2d5cd311407189409e976144b3fbd1e9ff42be77bb72be4f3eb"}
Oct 30 21:55:11.154398 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:11.153974    4412 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e2aaf8b0-d53f-4452-92f6-170dd6cb201a","Type":"ContainerStarted","Data":"234db1c1943fddeaa81ac32efa689d00626366720507aa507efa873548b5a8fb"}
Oct 30 21:55:11.154398 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:11.154006    4412 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e2aaf8b0-d53f-4452-92f6-170dd6cb201a","Type":"ContainerStarted","Data":"a8b791ebd56b85929e606cfd032f79c32a03d7e852ea6adbf22956cd07004466"}
Oct 30 21:55:11.154398 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:11.154030    4412 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e2aaf8b0-d53f-4452-92f6-170dd6cb201a","Type":"ContainerStarted","Data":"a1dce0456883bd1a2a369b189813ad48c097f9349f34514a4e01800ba1e25b08"}
Oct 30 21:55:11.154398 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:11.154052    4412 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e2aaf8b0-d53f-4452-92f6-170dd6cb201a","Type":"ContainerStarted","Data":"264fd8e605d9b15fe5622ef77cddf6cdf88e5f72fad5a99d8f0af266f5199220"}
Oct 30 21:55:11.282499 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:11.282445    4412 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=3.267877793 podStartE2EDuration="3.282432611s" totalImagesPullingTime="0s" totalInitContainerRuntime="14.554818ms" isStatefulPod=false podCreationTimestamp="2026-10-30 21:55:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:55:11.281617241 +0000 UTC m=+131.786289221" watchObservedRunningTime="2026-10-30 21:55:11.282432611 +0000 UTC m=+131.787104563"
Oct 30 21:55:13.730527 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:55:13.730500    4412 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:55:34.448922 ip-10-0-131-202 kubenswrapper[4412]: E1030 21:55:34.448880    4412 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bkhmz" containerName="extractor"
Oct 30 21:56:04.448160 ip-10-0-131-202 kubenswrapper[4412]: E1030 21:56:04.448099    4412 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bkhmz" containerName="extractor"
Oct 30 21:56:08.731264 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:56:08.731215    4412 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:56:08.747301 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:56:08.747277    4412 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:56:09.366493 ip-10-0-131-202 kubenswrapper[4412]: I1030 21:56:09.366467    4412 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:56:34.448462 ip-10-0-131-202 kubenswrapper[4412]: E1030 21:56:34.448425    4412 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bkhmz" containerName="extractor"
Oct 30 21:57:04.448160 ip-10-0-131-202 kubenswrapper[4412]: E1030 21:57:04.448107    4412 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bkhmz" containerName="extractor"
Oct 30 21:57:34.447573 ip-10-0-131-202 kubenswrapper[4412]: E1030 21:57:34.447536    4412 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bkhmz" containerName="extractor"
Oct 30 21:58:04.447786 ip-10-0-131-202 kubenswrapper[4412]: E1030 21:58:04.447744    4412 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bkhmz" containerName="extractor"
Oct 30 21:58:34.447608 ip-10-0-131-202 kubenswrapper[4412]: E1030 21:58:34.447528    4412 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bkhmz" containerName="extractor"
Oct 30 21:59:04.447998 ip-10-0-131-202 kubenswrapper[4412]: E1030 21:59:04.447958    4412 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bkhmz" containerName="extractor"
Oct 30 21:59:34.448058 ip-10-0-131-202 kubenswrapper[4412]: E1030 21:59:34.448020    4412 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bkhmz" containerName="extractor"
Oct 30 22:00:04.448412 ip-10-0-131-202 kubenswrapper[4412]: E1030 22:00:04.448369    4412 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bkhmz" containerName="extractor"
Oct 30 22:00:34.448988 ip-10-0-131-202 kubenswrapper[4412]: E1030 22:00:34.448952    4412 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bkhmz" containerName="extractor"
Oct 30 22:01:04.448232 ip-10-0-131-202 kubenswrapper[4412]: E1030 22:01:04.448194    4412 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bkhmz" containerName="extractor"
Oct 30 22:01:34.448475 ip-10-0-131-202 kubenswrapper[4412]: E1030 22:01:34.448397    4412 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bkhmz" containerName="extractor"
Oct 30 22:02:04.274464 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:02:04.274432    4412 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kserve/s3-init-tslvd"]
Oct 30 22:02:04.277680 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:02:04.277665    4412 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/s3-init-tslvd"
Oct 30 22:02:04.365284 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:02:04.365258    4412 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4613ba0b-5591-4a38-afbb-ba8da5b44703-model-cache\") pod \"s3-init-tslvd\" (UID: \"4613ba0b-5591-4a38-afbb-ba8da5b44703\") " pod="kserve/s3-init-tslvd"
Oct 30 22:02:04.365445 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:02:04.365293    4412 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t495c\" (UniqueName: \"kubernetes.io/projected/4613ba0b-5591-4a38-afbb-ba8da5b44703-kube-api-access-t495c\") pod \"s3-init-tslvd\" (UID: \"4613ba0b-5591-4a38-afbb-ba8da5b44703\") " pod="kserve/s3-init-tslvd"
Oct 30 22:02:04.370656 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:02:04.370630    4412 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"hf-token\""
Oct 30 22:02:04.370656 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:02:04.370630    4412 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"openshift-service-ca.crt\""
Oct 30 22:02:04.370826 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:02:04.370632    4412 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"default-dockercfg-p47c2\""
Oct 30 22:02:04.370826 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:02:04.370640    4412 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"mlpipeline-s3-artifact\""
Oct 30 22:02:04.370826 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:02:04.370653    4412 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"kube-root-ca.crt\""
Oct 30 22:02:04.371700 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:02:04.371682    4412 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kserve/s3-init-tslvd"]
Oct 30 22:02:04.447818 ip-10-0-131-202 kubenswrapper[4412]: E1030 22:02:04.447783    4412 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bkhmz" containerName="extractor"
Oct 30 22:02:04.466238 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:02:04.466218    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4613ba0b-5591-4a38-afbb-ba8da5b44703-model-cache\") pod \"s3-init-tslvd\" (UID: \"4613ba0b-5591-4a38-afbb-ba8da5b44703\") " pod="kserve/s3-init-tslvd"
Oct 30 22:02:04.466371 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:02:04.466250    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t495c\" (UniqueName: \"kubernetes.io/projected/4613ba0b-5591-4a38-afbb-ba8da5b44703-kube-api-access-t495c\") pod \"s3-init-tslvd\" (UID: \"4613ba0b-5591-4a38-afbb-ba8da5b44703\") " pod="kserve/s3-init-tslvd"
Oct 30 22:02:04.466582 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:02:04.466562    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4613ba0b-5591-4a38-afbb-ba8da5b44703-model-cache\") pod \"s3-init-tslvd\" (UID: \"4613ba0b-5591-4a38-afbb-ba8da5b44703\") " pod="kserve/s3-init-tslvd"
Oct 30 22:02:04.523755 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:02:04.523734    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t495c\" (UniqueName: \"kubernetes.io/projected/4613ba0b-5591-4a38-afbb-ba8da5b44703-kube-api-access-t495c\") pod \"s3-init-tslvd\" (UID: \"4613ba0b-5591-4a38-afbb-ba8da5b44703\") " pod="kserve/s3-init-tslvd"
Oct 30 22:02:04.606339 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:02:04.606280    4412 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/s3-init-tslvd"
Oct 30 22:02:04.974789 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:02:04.974767    4412 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kserve/s3-init-tslvd"]
Oct 30 22:02:04.977421 ip-10-0-131-202 kubenswrapper[4412]: W1030 22:02:04.977393    4412 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4613ba0b_5591_4a38_afbb_ba8da5b44703.slice/crio-26f0a478f93782d9d693640974b6f03aeadd95d08daa1c4876ce4de0c785b7c1 WatchSource:0}: Error finding container 26f0a478f93782d9d693640974b6f03aeadd95d08daa1c4876ce4de0c785b7c1: Status 404 returned error can't find the container with id 26f0a478f93782d9d693640974b6f03aeadd95d08daa1c4876ce4de0c785b7c1
Oct 30 22:02:04.979240 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:02:04.979225    4412 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 22:02:05.550859 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:02:05.550824    4412 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kserve/s3-init-tslvd" event={"ID":"4613ba0b-5591-4a38-afbb-ba8da5b44703","Type":"ContainerStarted","Data":"26f0a478f93782d9d693640974b6f03aeadd95d08daa1c4876ce4de0c785b7c1"}
Oct 30 22:02:13.582186 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:02:13.582147    4412 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kserve/s3-init-tslvd" event={"ID":"4613ba0b-5591-4a38-afbb-ba8da5b44703","Type":"ContainerStarted","Data":"714f78ac02c839fa396762c4ecc35519a4cd8e9f8bd81a976fc2196dc94d7890"}
Oct 30 22:02:15.590452 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:02:15.590365    4412 generic.go:356] "Generic (PLEG): container finished" podID="4613ba0b-5591-4a38-afbb-ba8da5b44703" containerID="714f78ac02c839fa396762c4ecc35519a4cd8e9f8bd81a976fc2196dc94d7890" exitCode=0
Oct 30 22:02:15.590452 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:02:15.590425    4412 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kserve/s3-init-tslvd" event={"ID":"4613ba0b-5591-4a38-afbb-ba8da5b44703","Type":"ContainerDied","Data":"714f78ac02c839fa396762c4ecc35519a4cd8e9f8bd81a976fc2196dc94d7890"}
Oct 30 22:02:20.612974 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:02:20.612944    4412 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kserve/s3-init-tslvd" event={"ID":"4613ba0b-5591-4a38-afbb-ba8da5b44703","Type":"ContainerStarted","Data":"9ea2f07618f6206b08e6b6255af8b9f9c0ee31a21a05865b6655f660e743ffbc"}
Oct 30 22:02:20.731188 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:02:20.731103    4412 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kserve/s3-init-tslvd" podStartSLOduration=1.544173945 podStartE2EDuration="16.731087246s" totalImagesPullingTime="12.9966049s" totalInitContainerRuntime="2.190308401s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:02:04 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:02:20.731052807 +0000 UTC m=+561.235724759" watchObservedRunningTime="2026-10-30 22:02:20.731087246 +0000 UTC m=+561.235759200"
Oct 30 22:02:34.447383 ip-10-0-131-202 kubenswrapper[4412]: E1030 22:02:34.447349    4412 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bkhmz" containerName="extractor"
Oct 30 22:03:04.449613 ip-10-0-131-202 kubenswrapper[4412]: E1030 22:03:04.449569    4412 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bkhmz" containerName="extractor"
Oct 30 22:03:25.840613 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:03:25.840523    4412 generic.go:356] "Generic (PLEG): container finished" podID="4613ba0b-5591-4a38-afbb-ba8da5b44703" containerID="9ea2f07618f6206b08e6b6255af8b9f9c0ee31a21a05865b6655f660e743ffbc" exitCode=0
Oct 30 22:03:25.840613 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:03:25.840564    4412 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kserve/s3-init-tslvd" event={"ID":"4613ba0b-5591-4a38-afbb-ba8da5b44703","Type":"ContainerDied","Data":"9ea2f07618f6206b08e6b6255af8b9f9c0ee31a21a05865b6655f660e743ffbc"}
Oct 30 22:03:26.962376 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:03:26.962354    4412 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve/s3-init-tslvd"
Oct 30 22:03:27.088786 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:03:27.088753    4412 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4613ba0b-5591-4a38-afbb-ba8da5b44703-kube-api-access-t495c\" (UniqueName: \"kubernetes.io/projected/4613ba0b-5591-4a38-afbb-ba8da5b44703-kube-api-access-t495c\") pod \"4613ba0b-5591-4a38-afbb-ba8da5b44703\" (UID: \"4613ba0b-5591-4a38-afbb-ba8da5b44703\") "
Oct 30 22:03:27.088978 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:03:27.088911    4412 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4613ba0b-5591-4a38-afbb-ba8da5b44703-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4613ba0b-5591-4a38-afbb-ba8da5b44703-model-cache\") pod \"4613ba0b-5591-4a38-afbb-ba8da5b44703\" (UID: \"4613ba0b-5591-4a38-afbb-ba8da5b44703\") "
Oct 30 22:03:27.091073 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:03:27.091045    4412 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4613ba0b-5591-4a38-afbb-ba8da5b44703-kube-api-access-t495c" pod "4613ba0b-5591-4a38-afbb-ba8da5b44703" (UID: "4613ba0b-5591-4a38-afbb-ba8da5b44703"). InnerVolumeSpecName "kube-api-access-t495c". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:03:27.111702 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:03:27.111673    4412 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4613ba0b-5591-4a38-afbb-ba8da5b44703-model-cache" pod "4613ba0b-5591-4a38-afbb-ba8da5b44703" (UID: "4613ba0b-5591-4a38-afbb-ba8da5b44703"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:03:27.189749 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:03:27.189716    4412 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4613ba0b-5591-4a38-afbb-ba8da5b44703-model-cache\") on node \"ip-10-0-131-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:03:27.189749 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:03:27.189746    4412 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t495c\" (UniqueName: \"kubernetes.io/projected/4613ba0b-5591-4a38-afbb-ba8da5b44703-kube-api-access-t495c\") on node \"ip-10-0-131-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:03:27.849664 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:03:27.849636    4412 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve/s3-init-tslvd"
Oct 30 22:03:27.849857 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:03:27.849623    4412 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kserve/s3-init-tslvd" event={"ID":"4613ba0b-5591-4a38-afbb-ba8da5b44703","Type":"ContainerDied","Data":"26f0a478f93782d9d693640974b6f03aeadd95d08daa1c4876ce4de0c785b7c1"}
Oct 30 22:03:27.849857 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:03:27.849763    4412 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="26f0a478f93782d9d693640974b6f03aeadd95d08daa1c4876ce4de0c785b7c1"
Oct 30 22:03:34.448846 ip-10-0-131-202 kubenswrapper[4412]: E1030 22:03:34.448807    4412 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bkhmz" containerName="extractor"
Oct 30 22:04:04.027944 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:04:04.027911    4412 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kserve/s3-tls-init-custom-72bwz"]
Oct 30 22:04:04.028426 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:04:04.028219    4412 cpu_manager.go:414] "containerMap: removing container" podUID="4613ba0b-5591-4a38-afbb-ba8da5b44703" containerName="s3-init"
Oct 30 22:04:04.028426 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:04:04.028230    4412 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4613ba0b-5591-4a38-afbb-ba8da5b44703" containerName="s3-init"
Oct 30 22:04:04.028426 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:04:04.028250    4412 cpu_manager.go:414] "containerMap: removing container" podUID="4613ba0b-5591-4a38-afbb-ba8da5b44703" containerName="download-hf-model"
Oct 30 22:04:04.028426 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:04:04.028255    4412 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4613ba0b-5591-4a38-afbb-ba8da5b44703" containerName="download-hf-model"
Oct 30 22:04:04.028426 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:04:04.028298    4412 memory_manager.go:365] "RemoveStaleState removing state" podUID="4613ba0b-5591-4a38-afbb-ba8da5b44703" containerName="s3-init"
Oct 30 22:04:04.029961 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:04:04.029946    4412 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/s3-tls-init-custom-72bwz"
Oct 30 22:04:04.083401 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:04:04.083373    4412 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"hf-token\""
Oct 30 22:04:04.083401 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:04:04.083371    4412 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"seaweedfs-tls-custom-artifact\""
Oct 30 22:04:04.133933 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:04:04.133875    4412 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"default-dockercfg-p47c2\""
Oct 30 22:04:04.133933 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:04:04.133887    4412 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"kube-root-ca.crt\""
Oct 30 22:04:04.133933 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:04:04.133892    4412 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"openshift-service-ca.crt\""
Oct 30 22:04:04.134961 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:04:04.134944    4412 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kserve/s3-tls-init-custom-72bwz"]
Oct 30 22:04:04.188611 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:04:04.188584    4412 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/44315fd5-9953-4e37-8028-be99ed553b92-model-cache\") pod \"s3-tls-init-custom-72bwz\" (UID: \"44315fd5-9953-4e37-8028-be99ed553b92\") " pod="kserve/s3-tls-init-custom-72bwz"
Oct 30 22:04:04.188725 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:04:04.188617    4412 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vrn6w\" (UniqueName: \"kubernetes.io/projected/44315fd5-9953-4e37-8028-be99ed553b92-kube-api-access-vrn6w\") pod \"s3-tls-init-custom-72bwz\" (UID: \"44315fd5-9953-4e37-8028-be99ed553b92\") " pod="kserve/s3-tls-init-custom-72bwz"
Oct 30 22:04:04.289861 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:04:04.289799    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/44315fd5-9953-4e37-8028-be99ed553b92-model-cache\") pod \"s3-tls-init-custom-72bwz\" (UID: \"44315fd5-9953-4e37-8028-be99ed553b92\") " pod="kserve/s3-tls-init-custom-72bwz"
Oct 30 22:04:04.289861 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:04:04.289834    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vrn6w\" (UniqueName: \"kubernetes.io/projected/44315fd5-9953-4e37-8028-be99ed553b92-kube-api-access-vrn6w\") pod \"s3-tls-init-custom-72bwz\" (UID: \"44315fd5-9953-4e37-8028-be99ed553b92\") " pod="kserve/s3-tls-init-custom-72bwz"
Oct 30 22:04:04.290175 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:04:04.290158    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/44315fd5-9953-4e37-8028-be99ed553b92-model-cache\") pod \"s3-tls-init-custom-72bwz\" (UID: \"44315fd5-9953-4e37-8028-be99ed553b92\") " pod="kserve/s3-tls-init-custom-72bwz"
Oct 30 22:04:04.348951 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:04:04.348917    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vrn6w\" (UniqueName: \"kubernetes.io/projected/44315fd5-9953-4e37-8028-be99ed553b92-kube-api-access-vrn6w\") pod \"s3-tls-init-custom-72bwz\" (UID: \"44315fd5-9953-4e37-8028-be99ed553b92\") " pod="kserve/s3-tls-init-custom-72bwz"
Oct 30 22:04:04.357140 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:04:04.357097    4412 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/s3-tls-init-custom-72bwz"
Oct 30 22:04:04.449345 ip-10-0-131-202 kubenswrapper[4412]: E1030 22:04:04.449311    4412 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bkhmz" containerName="extractor"
Oct 30 22:04:04.724526 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:04:04.724379    4412 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kserve/s3-tls-init-custom-72bwz"]
Oct 30 22:04:04.727302 ip-10-0-131-202 kubenswrapper[4412]: W1030 22:04:04.727274    4412 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod44315fd5_9953_4e37_8028_be99ed553b92.slice/crio-52570029a0399dbd3e0ac853def4a572211665fcb17dc61e7e2bd8c05d71db55 WatchSource:0}: Error finding container 52570029a0399dbd3e0ac853def4a572211665fcb17dc61e7e2bd8c05d71db55: Status 404 returned error can't find the container with id 52570029a0399dbd3e0ac853def4a572211665fcb17dc61e7e2bd8c05d71db55
Oct 30 22:04:04.984717 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:04:04.984630    4412 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kserve/s3-tls-init-custom-72bwz" event={"ID":"44315fd5-9953-4e37-8028-be99ed553b92","Type":"ContainerStarted","Data":"894e92f9a108f8a10a4892e57faf1b61aa3afdb14fd10d42cf9bcb21dcaf6b49"}
Oct 30 22:04:04.984717 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:04:04.984669    4412 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kserve/s3-tls-init-custom-72bwz" event={"ID":"44315fd5-9953-4e37-8028-be99ed553b92","Type":"ContainerStarted","Data":"52570029a0399dbd3e0ac853def4a572211665fcb17dc61e7e2bd8c05d71db55"}
Oct 30 22:04:07.996391 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:04:07.996364    4412 generic.go:356] "Generic (PLEG): container finished" podID="44315fd5-9953-4e37-8028-be99ed553b92" containerID="894e92f9a108f8a10a4892e57faf1b61aa3afdb14fd10d42cf9bcb21dcaf6b49" exitCode=0
Oct 30 22:04:07.996696 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:04:07.996436    4412 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kserve/s3-tls-init-custom-72bwz" event={"ID":"44315fd5-9953-4e37-8028-be99ed553b92","Type":"ContainerDied","Data":"894e92f9a108f8a10a4892e57faf1b61aa3afdb14fd10d42cf9bcb21dcaf6b49"}
Oct 30 22:04:09.001805 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:04:09.001772    4412 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kserve/s3-tls-init-custom-72bwz" event={"ID":"44315fd5-9953-4e37-8028-be99ed553b92","Type":"ContainerStarted","Data":"750636c802ba407cc1acf6364d8cdd7944ebedb43120e7497d8c4ed44db621da"}
Oct 30 22:04:09.118427 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:04:09.118372    4412 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kserve/s3-tls-init-custom-72bwz" podStartSLOduration=2.930891092 podStartE2EDuration="6.118358797s" totalImagesPullingTime="0s" totalInitContainerRuntime="3.187467705s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:04:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:04:09.117820707 +0000 UTC m=+669.622492661" watchObservedRunningTime="2026-10-30 22:04:09.118358797 +0000 UTC m=+669.623030752"
Oct 30 22:04:15.025278 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:04:15.025242    4412 generic.go:356] "Generic (PLEG): container finished" podID="44315fd5-9953-4e37-8028-be99ed553b92" containerID="750636c802ba407cc1acf6364d8cdd7944ebedb43120e7497d8c4ed44db621da" exitCode=0
Oct 30 22:04:15.025661 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:04:15.025313    4412 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kserve/s3-tls-init-custom-72bwz" event={"ID":"44315fd5-9953-4e37-8028-be99ed553b92","Type":"ContainerDied","Data":"750636c802ba407cc1acf6364d8cdd7944ebedb43120e7497d8c4ed44db621da"}
Oct 30 22:04:16.156538 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:04:16.156514    4412 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve/s3-tls-init-custom-72bwz"
Oct 30 22:04:16.187675 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:04:16.187651    4412 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/44315fd5-9953-4e37-8028-be99ed553b92-kube-api-access-vrn6w\" (UniqueName: \"kubernetes.io/projected/44315fd5-9953-4e37-8028-be99ed553b92-kube-api-access-vrn6w\") pod \"44315fd5-9953-4e37-8028-be99ed553b92\" (UID: \"44315fd5-9953-4e37-8028-be99ed553b92\") "
Oct 30 22:04:16.187787 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:04:16.187699    4412 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/44315fd5-9953-4e37-8028-be99ed553b92-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/44315fd5-9953-4e37-8028-be99ed553b92-model-cache\") pod \"44315fd5-9953-4e37-8028-be99ed553b92\" (UID: \"44315fd5-9953-4e37-8028-be99ed553b92\") "
Oct 30 22:04:16.189787 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:04:16.189764    4412 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/44315fd5-9953-4e37-8028-be99ed553b92-kube-api-access-vrn6w" pod "44315fd5-9953-4e37-8028-be99ed553b92" (UID: "44315fd5-9953-4e37-8028-be99ed553b92"). InnerVolumeSpecName "kube-api-access-vrn6w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:04:16.278258 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:04:16.278220    4412 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/44315fd5-9953-4e37-8028-be99ed553b92-model-cache" pod "44315fd5-9953-4e37-8028-be99ed553b92" (UID: "44315fd5-9953-4e37-8028-be99ed553b92"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:04:16.288278 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:04:16.288213    4412 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vrn6w\" (UniqueName: \"kubernetes.io/projected/44315fd5-9953-4e37-8028-be99ed553b92-kube-api-access-vrn6w\") on node \"ip-10-0-131-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:04:16.288278 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:04:16.288244    4412 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/44315fd5-9953-4e37-8028-be99ed553b92-model-cache\") on node \"ip-10-0-131-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:04:17.033206 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:04:17.033174    4412 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kserve/s3-tls-init-custom-72bwz" event={"ID":"44315fd5-9953-4e37-8028-be99ed553b92","Type":"ContainerDied","Data":"52570029a0399dbd3e0ac853def4a572211665fcb17dc61e7e2bd8c05d71db55"}
Oct 30 22:04:17.033374 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:04:17.033212    4412 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="52570029a0399dbd3e0ac853def4a572211665fcb17dc61e7e2bd8c05d71db55"
Oct 30 22:04:17.033374 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:04:17.033218    4412 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve/s3-tls-init-custom-72bwz"
Oct 30 22:04:21.435818 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:04:21.435785    4412 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kserve/s3-tls-init-serving-lqcx2"]
Oct 30 22:04:21.436215 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:04:21.436086    4412 cpu_manager.go:414] "containerMap: removing container" podUID="44315fd5-9953-4e37-8028-be99ed553b92" containerName="s3-tls-init-custom"
Oct 30 22:04:21.436215 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:04:21.436096    4412 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="44315fd5-9953-4e37-8028-be99ed553b92" containerName="s3-tls-init-custom"
Oct 30 22:04:21.436215 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:04:21.436134    4412 cpu_manager.go:414] "containerMap: removing container" podUID="44315fd5-9953-4e37-8028-be99ed553b92" containerName="download-hf-model"
Oct 30 22:04:21.436215 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:04:21.436139    4412 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="44315fd5-9953-4e37-8028-be99ed553b92" containerName="download-hf-model"
Oct 30 22:04:21.436215 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:04:21.436181    4412 memory_manager.go:365] "RemoveStaleState removing state" podUID="44315fd5-9953-4e37-8028-be99ed553b92" containerName="s3-tls-init-custom"
Oct 30 22:04:21.438956 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:04:21.438939    4412 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/s3-tls-init-serving-lqcx2"
Oct 30 22:04:21.491715 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:04:21.491695    4412 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"hf-token\""
Oct 30 22:04:21.491863 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:04:21.491739    4412 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"seaweedfs-tls-serving-artifact\""
Oct 30 22:04:21.530197 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:04:21.530169    4412 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s5css\" (UniqueName: \"kubernetes.io/projected/00d3c368-b282-45a4-a6a0-107269a7cf37-kube-api-access-s5css\") pod \"s3-tls-init-serving-lqcx2\" (UID: \"00d3c368-b282-45a4-a6a0-107269a7cf37\") " pod="kserve/s3-tls-init-serving-lqcx2"
Oct 30 22:04:21.530308 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:04:21.530218    4412 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/00d3c368-b282-45a4-a6a0-107269a7cf37-model-cache\") pod \"s3-tls-init-serving-lqcx2\" (UID: \"00d3c368-b282-45a4-a6a0-107269a7cf37\") " pod="kserve/s3-tls-init-serving-lqcx2"
Oct 30 22:04:21.542026 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:04:21.542003    4412 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"openshift-service-ca.crt\""
Oct 30 22:04:21.542171 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:04:21.542050    4412 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"kube-root-ca.crt\""
Oct 30 22:04:21.542171 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:04:21.542091    4412 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"default-dockercfg-p47c2\""
Oct 30 22:04:21.543026 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:04:21.543009    4412 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kserve/s3-tls-init-serving-lqcx2"]
Oct 30 22:04:21.631359 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:04:21.631338    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s5css\" (UniqueName: \"kubernetes.io/projected/00d3c368-b282-45a4-a6a0-107269a7cf37-kube-api-access-s5css\") pod \"s3-tls-init-serving-lqcx2\" (UID: \"00d3c368-b282-45a4-a6a0-107269a7cf37\") " pod="kserve/s3-tls-init-serving-lqcx2"
Oct 30 22:04:21.631469 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:04:21.631370    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/00d3c368-b282-45a4-a6a0-107269a7cf37-model-cache\") pod \"s3-tls-init-serving-lqcx2\" (UID: \"00d3c368-b282-45a4-a6a0-107269a7cf37\") " pod="kserve/s3-tls-init-serving-lqcx2"
Oct 30 22:04:21.631674 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:04:21.631660    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/00d3c368-b282-45a4-a6a0-107269a7cf37-model-cache\") pod \"s3-tls-init-serving-lqcx2\" (UID: \"00d3c368-b282-45a4-a6a0-107269a7cf37\") " pod="kserve/s3-tls-init-serving-lqcx2"
Oct 30 22:04:21.689453 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:04:21.689391    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s5css\" (UniqueName: \"kubernetes.io/projected/00d3c368-b282-45a4-a6a0-107269a7cf37-kube-api-access-s5css\") pod \"s3-tls-init-serving-lqcx2\" (UID: \"00d3c368-b282-45a4-a6a0-107269a7cf37\") " pod="kserve/s3-tls-init-serving-lqcx2"
Oct 30 22:04:21.762380 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:04:21.762360    4412 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/s3-tls-init-serving-lqcx2"
Oct 30 22:04:22.133015 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:04:22.132961    4412 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kserve/s3-tls-init-serving-lqcx2"]
Oct 30 22:04:22.135287 ip-10-0-131-202 kubenswrapper[4412]: W1030 22:04:22.135259    4412 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod00d3c368_b282_45a4_a6a0_107269a7cf37.slice/crio-0049cde41b296300a1558b651970566ac85bca1fad298bcba4fd2a1546ba91a4 WatchSource:0}: Error finding container 0049cde41b296300a1558b651970566ac85bca1fad298bcba4fd2a1546ba91a4: Status 404 returned error can't find the container with id 0049cde41b296300a1558b651970566ac85bca1fad298bcba4fd2a1546ba91a4
Oct 30 22:04:23.055548 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:04:23.055505    4412 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kserve/s3-tls-init-serving-lqcx2" event={"ID":"00d3c368-b282-45a4-a6a0-107269a7cf37","Type":"ContainerStarted","Data":"76afad4d70e3ac7d2b1345219cfb65f14d58e39a6905acbf5737dd6ae872b78c"}
Oct 30 22:04:23.055548 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:04:23.055545    4412 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kserve/s3-tls-init-serving-lqcx2" event={"ID":"00d3c368-b282-45a4-a6a0-107269a7cf37","Type":"ContainerStarted","Data":"0049cde41b296300a1558b651970566ac85bca1fad298bcba4fd2a1546ba91a4"}
Oct 30 22:04:26.067535 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:04:26.067503    4412 generic.go:356] "Generic (PLEG): container finished" podID="00d3c368-b282-45a4-a6a0-107269a7cf37" containerID="76afad4d70e3ac7d2b1345219cfb65f14d58e39a6905acbf5737dd6ae872b78c" exitCode=0
Oct 30 22:04:26.068005 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:04:26.067550    4412 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kserve/s3-tls-init-serving-lqcx2" event={"ID":"00d3c368-b282-45a4-a6a0-107269a7cf37","Type":"ContainerDied","Data":"76afad4d70e3ac7d2b1345219cfb65f14d58e39a6905acbf5737dd6ae872b78c"}
Oct 30 22:04:27.073435 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:04:27.073403    4412 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kserve/s3-tls-init-serving-lqcx2" event={"ID":"00d3c368-b282-45a4-a6a0-107269a7cf37","Type":"ContainerStarted","Data":"2439549aa6631c1d64ec3c8e42bb1596b0dd38c492ef25b0e0dfc45563a99cb9"}
Oct 30 22:04:27.190926 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:04:27.190875    4412 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kserve/s3-tls-init-serving-lqcx2" podStartSLOduration=2.988842749 podStartE2EDuration="6.190861574s" totalImagesPullingTime="0s" totalInitContainerRuntime="3.202018825s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:04:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:04:27.19018257 +0000 UTC m=+687.694854523" watchObservedRunningTime="2026-10-30 22:04:27.190861574 +0000 UTC m=+687.695533526"
Oct 30 22:04:33.096593 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:04:33.096561    4412 generic.go:356] "Generic (PLEG): container finished" podID="00d3c368-b282-45a4-a6a0-107269a7cf37" containerID="2439549aa6631c1d64ec3c8e42bb1596b0dd38c492ef25b0e0dfc45563a99cb9" exitCode=0
Oct 30 22:04:33.096926 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:04:33.096634    4412 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kserve/s3-tls-init-serving-lqcx2" event={"ID":"00d3c368-b282-45a4-a6a0-107269a7cf37","Type":"ContainerDied","Data":"2439549aa6631c1d64ec3c8e42bb1596b0dd38c492ef25b0e0dfc45563a99cb9"}
Oct 30 22:04:34.227179 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:04:34.227155    4412 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve/s3-tls-init-serving-lqcx2"
Oct 30 22:04:34.328559 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:04:34.328530    4412 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/00d3c368-b282-45a4-a6a0-107269a7cf37-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/00d3c368-b282-45a4-a6a0-107269a7cf37-model-cache\") pod \"00d3c368-b282-45a4-a6a0-107269a7cf37\" (UID: \"00d3c368-b282-45a4-a6a0-107269a7cf37\") "
Oct 30 22:04:34.328559 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:04:34.328563    4412 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/00d3c368-b282-45a4-a6a0-107269a7cf37-kube-api-access-s5css\" (UniqueName: \"kubernetes.io/projected/00d3c368-b282-45a4-a6a0-107269a7cf37-kube-api-access-s5css\") pod \"00d3c368-b282-45a4-a6a0-107269a7cf37\" (UID: \"00d3c368-b282-45a4-a6a0-107269a7cf37\") "
Oct 30 22:04:34.330804 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:04:34.330771    4412 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/00d3c368-b282-45a4-a6a0-107269a7cf37-kube-api-access-s5css" pod "00d3c368-b282-45a4-a6a0-107269a7cf37" (UID: "00d3c368-b282-45a4-a6a0-107269a7cf37"). InnerVolumeSpecName "kube-api-access-s5css". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:04:34.349751 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:04:34.349724    4412 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/00d3c368-b282-45a4-a6a0-107269a7cf37-model-cache" pod "00d3c368-b282-45a4-a6a0-107269a7cf37" (UID: "00d3c368-b282-45a4-a6a0-107269a7cf37"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:04:34.430089 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:04:34.430018    4412 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/00d3c368-b282-45a4-a6a0-107269a7cf37-model-cache\") on node \"ip-10-0-131-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:04:34.430089 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:04:34.430041    4412 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s5css\" (UniqueName: \"kubernetes.io/projected/00d3c368-b282-45a4-a6a0-107269a7cf37-kube-api-access-s5css\") on node \"ip-10-0-131-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:04:34.447648 ip-10-0-131-202 kubenswrapper[4412]: E1030 22:04:34.447615    4412 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bkhmz" containerName="extractor"
Oct 30 22:04:35.105400 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:04:35.105369    4412 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kserve/s3-tls-init-serving-lqcx2" event={"ID":"00d3c368-b282-45a4-a6a0-107269a7cf37","Type":"ContainerDied","Data":"0049cde41b296300a1558b651970566ac85bca1fad298bcba4fd2a1546ba91a4"}
Oct 30 22:04:35.105400 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:04:35.105406    4412 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0049cde41b296300a1558b651970566ac85bca1fad298bcba4fd2a1546ba91a4"
Oct 30 22:04:35.105400 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:04:35.105404    4412 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve/s3-tls-init-serving-lqcx2"
Oct 30 22:04:57.201132 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:04:57.201095    4412 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-gw1-7476d02544/isvc-pmml-v2-kserve-predictor-ff9978f97-p7f27"]
Oct 30 22:04:57.201613 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:04:57.201418    4412 cpu_manager.go:414] "containerMap: removing container" podUID="00d3c368-b282-45a4-a6a0-107269a7cf37" containerName="s3-tls-init-serving"
Oct 30 22:04:57.201613 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:04:57.201428    4412 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="00d3c368-b282-45a4-a6a0-107269a7cf37" containerName="s3-tls-init-serving"
Oct 30 22:04:57.201613 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:04:57.201447    4412 cpu_manager.go:414] "containerMap: removing container" podUID="00d3c368-b282-45a4-a6a0-107269a7cf37" containerName="download-hf-model"
Oct 30 22:04:57.201613 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:04:57.201452    4412 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="00d3c368-b282-45a4-a6a0-107269a7cf37" containerName="download-hf-model"
Oct 30 22:04:57.201613 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:04:57.201494    4412 memory_manager.go:365] "RemoveStaleState removing state" podUID="00d3c368-b282-45a4-a6a0-107269a7cf37" containerName="s3-tls-init-serving"
Oct 30 22:04:57.203355 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:04:57.203338    4412 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-7476d02544/isvc-pmml-v2-kserve-predictor-ff9978f97-p7f27"
Oct 30 22:04:57.256562 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:04:57.256528    4412 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw1-7476d02544\"/\"isvc-pmml-v2-kserve-kube-rbac-proxy-sar-config\""
Oct 30 22:04:57.306951 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:04:57.306922    4412 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw1-7476d02544\"/\"openshift-service-ca.crt\""
Oct 30 22:04:57.306951 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:04:57.306925    4412 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw1-7476d02544\"/\"kube-root-ca.crt\""
Oct 30 22:04:57.306951 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:04:57.306955    4412 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw1-7476d02544\"/\"default-dockercfg-cxfkn\""
Oct 30 22:04:57.307226 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:04:57.306933    4412 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw1-7476d02544\"/\"odh-kserve-custom-ca-bundle\""
Oct 30 22:04:57.307226 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:04:57.306971    4412 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw1-7476d02544\"/\"seaweedfs-s3-creds\""
Oct 30 22:04:57.307226 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:04:57.307024    4412 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw1-7476d02544\"/\"isvc-pmml-v2-kserve-predictor-serving-cert\""
Oct 30 22:04:57.308231 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:04:57.308212    4412 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw1-7476d02544/isvc-pmml-v2-kserve-predictor-ff9978f97-p7f27"]
Oct 30 22:04:57.312009 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:04:57.311994    4412 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/3e9d15e2-b8d8-45f5-98c2-9b112fb7b31e-cabundle-cert\") pod \"isvc-pmml-v2-kserve-predictor-ff9978f97-p7f27\" (UID: \"3e9d15e2-b8d8-45f5-98c2-9b112fb7b31e\") " pod="e2e-gw1-7476d02544/isvc-pmml-v2-kserve-predictor-ff9978f97-p7f27"
Oct 30 22:04:57.312062 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:04:57.312023    4412 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/3e9d15e2-b8d8-45f5-98c2-9b112fb7b31e-proxy-tls\") pod \"isvc-pmml-v2-kserve-predictor-ff9978f97-p7f27\" (UID: \"3e9d15e2-b8d8-45f5-98c2-9b112fb7b31e\") " pod="e2e-gw1-7476d02544/isvc-pmml-v2-kserve-predictor-ff9978f97-p7f27"
Oct 30 22:04:57.312104 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:04:57.312064    4412 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/3e9d15e2-b8d8-45f5-98c2-9b112fb7b31e-kserve-provision-location\") pod \"isvc-pmml-v2-kserve-predictor-ff9978f97-p7f27\" (UID: \"3e9d15e2-b8d8-45f5-98c2-9b112fb7b31e\") " pod="e2e-gw1-7476d02544/isvc-pmml-v2-kserve-predictor-ff9978f97-p7f27"
Oct 30 22:04:57.312183 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:04:57.312168    4412 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-smpzz\" (UniqueName: \"kubernetes.io/projected/3e9d15e2-b8d8-45f5-98c2-9b112fb7b31e-kube-api-access-smpzz\") pod \"isvc-pmml-v2-kserve-predictor-ff9978f97-p7f27\" (UID: \"3e9d15e2-b8d8-45f5-98c2-9b112fb7b31e\") " pod="e2e-gw1-7476d02544/isvc-pmml-v2-kserve-predictor-ff9978f97-p7f27"
Oct 30 22:04:57.312226 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:04:57.312214    4412 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/3e9d15e2-b8d8-45f5-98c2-9b112fb7b31e-kube-rbac-proxy-sar-config\") pod \"isvc-pmml-v2-kserve-predictor-ff9978f97-p7f27\" (UID: \"3e9d15e2-b8d8-45f5-98c2-9b112fb7b31e\") " pod="e2e-gw1-7476d02544/isvc-pmml-v2-kserve-predictor-ff9978f97-p7f27"
Oct 30 22:04:57.412718 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:04:57.412691    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/3e9d15e2-b8d8-45f5-98c2-9b112fb7b31e-kube-rbac-proxy-sar-config\") pod \"isvc-pmml-v2-kserve-predictor-ff9978f97-p7f27\" (UID: \"3e9d15e2-b8d8-45f5-98c2-9b112fb7b31e\") " pod="e2e-gw1-7476d02544/isvc-pmml-v2-kserve-predictor-ff9978f97-p7f27"
Oct 30 22:04:57.412877 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:04:57.412732    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/3e9d15e2-b8d8-45f5-98c2-9b112fb7b31e-cabundle-cert\") pod \"isvc-pmml-v2-kserve-predictor-ff9978f97-p7f27\" (UID: \"3e9d15e2-b8d8-45f5-98c2-9b112fb7b31e\") " pod="e2e-gw1-7476d02544/isvc-pmml-v2-kserve-predictor-ff9978f97-p7f27"
Oct 30 22:04:57.412877 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:04:57.412753    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/3e9d15e2-b8d8-45f5-98c2-9b112fb7b31e-proxy-tls\") pod \"isvc-pmml-v2-kserve-predictor-ff9978f97-p7f27\" (UID: \"3e9d15e2-b8d8-45f5-98c2-9b112fb7b31e\") " pod="e2e-gw1-7476d02544/isvc-pmml-v2-kserve-predictor-ff9978f97-p7f27"
Oct 30 22:04:57.412877 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:04:57.412794    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/3e9d15e2-b8d8-45f5-98c2-9b112fb7b31e-kserve-provision-location\") pod \"isvc-pmml-v2-kserve-predictor-ff9978f97-p7f27\" (UID: \"3e9d15e2-b8d8-45f5-98c2-9b112fb7b31e\") " pod="e2e-gw1-7476d02544/isvc-pmml-v2-kserve-predictor-ff9978f97-p7f27"
Oct 30 22:04:57.412877 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:04:57.412812    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-smpzz\" (UniqueName: \"kubernetes.io/projected/3e9d15e2-b8d8-45f5-98c2-9b112fb7b31e-kube-api-access-smpzz\") pod \"isvc-pmml-v2-kserve-predictor-ff9978f97-p7f27\" (UID: \"3e9d15e2-b8d8-45f5-98c2-9b112fb7b31e\") " pod="e2e-gw1-7476d02544/isvc-pmml-v2-kserve-predictor-ff9978f97-p7f27"
Oct 30 22:04:57.413251 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:04:57.413228    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/3e9d15e2-b8d8-45f5-98c2-9b112fb7b31e-kserve-provision-location\") pod \"isvc-pmml-v2-kserve-predictor-ff9978f97-p7f27\" (UID: \"3e9d15e2-b8d8-45f5-98c2-9b112fb7b31e\") " pod="e2e-gw1-7476d02544/isvc-pmml-v2-kserve-predictor-ff9978f97-p7f27"
Oct 30 22:04:57.413450 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:04:57.413431    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/3e9d15e2-b8d8-45f5-98c2-9b112fb7b31e-cabundle-cert\") pod \"isvc-pmml-v2-kserve-predictor-ff9978f97-p7f27\" (UID: \"3e9d15e2-b8d8-45f5-98c2-9b112fb7b31e\") " pod="e2e-gw1-7476d02544/isvc-pmml-v2-kserve-predictor-ff9978f97-p7f27"
Oct 30 22:04:57.413517 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:04:57.413472    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/3e9d15e2-b8d8-45f5-98c2-9b112fb7b31e-kube-rbac-proxy-sar-config\") pod \"isvc-pmml-v2-kserve-predictor-ff9978f97-p7f27\" (UID: \"3e9d15e2-b8d8-45f5-98c2-9b112fb7b31e\") " pod="e2e-gw1-7476d02544/isvc-pmml-v2-kserve-predictor-ff9978f97-p7f27"
Oct 30 22:04:57.415373 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:04:57.415356    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/3e9d15e2-b8d8-45f5-98c2-9b112fb7b31e-proxy-tls\") pod \"isvc-pmml-v2-kserve-predictor-ff9978f97-p7f27\" (UID: \"3e9d15e2-b8d8-45f5-98c2-9b112fb7b31e\") " pod="e2e-gw1-7476d02544/isvc-pmml-v2-kserve-predictor-ff9978f97-p7f27"
Oct 30 22:04:57.471769 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:04:57.471743    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-smpzz\" (UniqueName: \"kubernetes.io/projected/3e9d15e2-b8d8-45f5-98c2-9b112fb7b31e-kube-api-access-smpzz\") pod \"isvc-pmml-v2-kserve-predictor-ff9978f97-p7f27\" (UID: \"3e9d15e2-b8d8-45f5-98c2-9b112fb7b31e\") " pod="e2e-gw1-7476d02544/isvc-pmml-v2-kserve-predictor-ff9978f97-p7f27"
Oct 30 22:04:57.515723 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:04:57.515700    4412 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-7476d02544/isvc-pmml-v2-kserve-predictor-ff9978f97-p7f27"
Oct 30 22:04:57.937879 ip-10-0-131-202 kubenswrapper[4412]: W1030 22:04:57.937850    4412 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3e9d15e2_b8d8_45f5_98c2_9b112fb7b31e.slice/crio-3f67b6ab72ae9bc77b772bd0fa1a0b7972a54f4c099370959674e622cf89d51d WatchSource:0}: Error finding container 3f67b6ab72ae9bc77b772bd0fa1a0b7972a54f4c099370959674e622cf89d51d: Status 404 returned error can't find the container with id 3f67b6ab72ae9bc77b772bd0fa1a0b7972a54f4c099370959674e622cf89d51d
Oct 30 22:04:57.938224 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:04:57.938206    4412 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw1-7476d02544/isvc-pmml-v2-kserve-predictor-ff9978f97-p7f27"]
Oct 30 22:04:58.190282 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:04:58.190204    4412 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-7476d02544/isvc-pmml-v2-kserve-predictor-ff9978f97-p7f27" event={"ID":"3e9d15e2-b8d8-45f5-98c2-9b112fb7b31e","Type":"ContainerStarted","Data":"657df0d2d692f93f21897b694f6ef14c5e067a9a5cbae784523bf0ee972c4f8a"}
Oct 30 22:04:58.190282 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:04:58.190258    4412 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-7476d02544/isvc-pmml-v2-kserve-predictor-ff9978f97-p7f27" event={"ID":"3e9d15e2-b8d8-45f5-98c2-9b112fb7b31e","Type":"ContainerStarted","Data":"3f67b6ab72ae9bc77b772bd0fa1a0b7972a54f4c099370959674e622cf89d51d"}
Oct 30 22:05:02.207757 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:05:02.207665    4412 generic.go:356] "Generic (PLEG): container finished" podID="3e9d15e2-b8d8-45f5-98c2-9b112fb7b31e" containerID="657df0d2d692f93f21897b694f6ef14c5e067a9a5cbae784523bf0ee972c4f8a" exitCode=0
Oct 30 22:05:02.207757 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:05:02.207737    4412 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-7476d02544/isvc-pmml-v2-kserve-predictor-ff9978f97-p7f27" event={"ID":"3e9d15e2-b8d8-45f5-98c2-9b112fb7b31e","Type":"ContainerDied","Data":"657df0d2d692f93f21897b694f6ef14c5e067a9a5cbae784523bf0ee972c4f8a"}
Oct 30 22:05:04.467044 ip-10-0-131-202 kubenswrapper[4412]: E1030 22:05:04.466958    4412 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bkhmz" containerName="extractor"
Oct 30 22:05:11.249858 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:05:11.249832    4412 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-7476d02544/isvc-pmml-v2-kserve-predictor-ff9978f97-p7f27" event={"ID":"3e9d15e2-b8d8-45f5-98c2-9b112fb7b31e","Type":"ContainerStarted","Data":"3ca54bbb04c1327a7698d972263c71372ad36cc3e12dbe8521239b7402777c81"}
Oct 30 22:05:15.269028 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:05:15.268988    4412 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-7476d02544/isvc-pmml-v2-kserve-predictor-ff9978f97-p7f27" event={"ID":"3e9d15e2-b8d8-45f5-98c2-9b112fb7b31e","Type":"ContainerStarted","Data":"9113dbab499819be8ff46510eecd467629a86caa54f81930ac138d55e1e55774"}
Oct 30 22:05:15.269441 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:05:15.269108    4412 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw1-7476d02544/isvc-pmml-v2-kserve-predictor-ff9978f97-p7f27"
Oct 30 22:05:15.451335 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:05:15.451284    4412 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-gw1-7476d02544/isvc-pmml-v2-kserve-predictor-ff9978f97-p7f27" podStartSLOduration=2.259999924 podStartE2EDuration="18.451270806s" totalImagesPullingTime="12.446871555s" totalInitContainerRuntime="3.744399327s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:04:57 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:05:15.450158735 +0000 UTC m=+735.954830689" watchObservedRunningTime="2026-10-30 22:05:15.451270806 +0000 UTC m=+735.955942758"
Oct 30 22:05:16.273106 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:05:16.273077    4412 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw1-7476d02544/isvc-pmml-v2-kserve-predictor-ff9978f97-p7f27"
Oct 30 22:05:16.274375 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:05:16.274352    4412 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-7476d02544/isvc-pmml-v2-kserve-predictor-ff9978f97-p7f27" podUID="3e9d15e2-b8d8-45f5-98c2-9b112fb7b31e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.20:8080: connect: connection refused"
Oct 30 22:05:17.276437 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:05:17.276402    4412 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-7476d02544/isvc-pmml-v2-kserve-predictor-ff9978f97-p7f27" podUID="3e9d15e2-b8d8-45f5-98c2-9b112fb7b31e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.20:8080: connect: connection refused"
Oct 30 22:05:22.281157 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:05:22.281109    4412 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw1-7476d02544/isvc-pmml-v2-kserve-predictor-ff9978f97-p7f27"
Oct 30 22:05:22.281646 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:05:22.281619    4412 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-7476d02544/isvc-pmml-v2-kserve-predictor-ff9978f97-p7f27" podUID="3e9d15e2-b8d8-45f5-98c2-9b112fb7b31e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.20:8080: connect: connection refused"
Oct 30 22:05:32.282032 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:05:32.281996    4412 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-7476d02544/isvc-pmml-v2-kserve-predictor-ff9978f97-p7f27" podUID="3e9d15e2-b8d8-45f5-98c2-9b112fb7b31e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.20:8080: connect: connection refused"
Oct 30 22:05:34.448206 ip-10-0-131-202 kubenswrapper[4412]: E1030 22:05:34.448170    4412 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bkhmz" containerName="extractor"
Oct 30 22:05:42.281683 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:05:42.281603    4412 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-7476d02544/isvc-pmml-v2-kserve-predictor-ff9978f97-p7f27" podUID="3e9d15e2-b8d8-45f5-98c2-9b112fb7b31e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.20:8080: connect: connection refused"
Oct 30 22:05:52.281959 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:05:52.281918    4412 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-7476d02544/isvc-pmml-v2-kserve-predictor-ff9978f97-p7f27" podUID="3e9d15e2-b8d8-45f5-98c2-9b112fb7b31e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.20:8080: connect: connection refused"
Oct 30 22:06:02.282514 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:06:02.282464    4412 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-7476d02544/isvc-pmml-v2-kserve-predictor-ff9978f97-p7f27" podUID="3e9d15e2-b8d8-45f5-98c2-9b112fb7b31e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.20:8080: connect: connection refused"
Oct 30 22:06:04.449086 ip-10-0-131-202 kubenswrapper[4412]: E1030 22:06:04.449045    4412 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bkhmz" containerName="extractor"
Oct 30 22:06:12.282503 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:06:12.282457    4412 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-7476d02544/isvc-pmml-v2-kserve-predictor-ff9978f97-p7f27" podUID="3e9d15e2-b8d8-45f5-98c2-9b112fb7b31e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.20:8080: connect: connection refused"
Oct 30 22:06:22.281911 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:06:22.281871    4412 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-7476d02544/isvc-pmml-v2-kserve-predictor-ff9978f97-p7f27" podUID="3e9d15e2-b8d8-45f5-98c2-9b112fb7b31e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.20:8080: connect: connection refused"
Oct 30 22:06:32.282212 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:06:32.282166    4412 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-7476d02544/isvc-pmml-v2-kserve-predictor-ff9978f97-p7f27" podUID="3e9d15e2-b8d8-45f5-98c2-9b112fb7b31e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.20:8080: connect: connection refused"
Oct 30 22:06:34.448716 ip-10-0-131-202 kubenswrapper[4412]: E1030 22:06:34.448679    4412 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bkhmz" containerName="extractor"
Oct 30 22:06:34.621683 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:06:34.621641    4412 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-7476d02544/isvc-pmml-v2-kserve-predictor-ff9978f97-p7f27" podUID="3e9d15e2-b8d8-45f5-98c2-9b112fb7b31e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.20:8080: connect: connection refused"
Oct 30 22:06:44.622880 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:06:44.622846    4412 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw1-7476d02544/isvc-pmml-v2-kserve-predictor-ff9978f97-p7f27"
Oct 30 22:06:58.145402 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:06:58.145371    4412 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw1-7476d02544/isvc-pmml-v2-kserve-predictor-ff9978f97-p7f27"]
Oct 30 22:06:58.145865 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:06:58.145595    4412 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw1-7476d02544/isvc-pmml-v2-kserve-predictor-ff9978f97-p7f27" podUID="3e9d15e2-b8d8-45f5-98c2-9b112fb7b31e" containerName="kserve-container" containerID="cri-o://3ca54bbb04c1327a7698d972263c71372ad36cc3e12dbe8521239b7402777c81" gracePeriod=30
Oct 30 22:06:58.145865 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:06:58.145647    4412 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw1-7476d02544/isvc-pmml-v2-kserve-predictor-ff9978f97-p7f27" podUID="3e9d15e2-b8d8-45f5-98c2-9b112fb7b31e" containerName="kube-rbac-proxy" containerID="cri-o://9113dbab499819be8ff46510eecd467629a86caa54f81930ac138d55e1e55774" gracePeriod=30
Oct 30 22:06:58.650826 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:06:58.650791    4412 generic.go:356] "Generic (PLEG): container finished" podID="3e9d15e2-b8d8-45f5-98c2-9b112fb7b31e" containerID="9113dbab499819be8ff46510eecd467629a86caa54f81930ac138d55e1e55774" exitCode=0
Oct 30 22:06:58.650996 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:06:58.650857    4412 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-7476d02544/isvc-pmml-v2-kserve-predictor-ff9978f97-p7f27" event={"ID":"3e9d15e2-b8d8-45f5-98c2-9b112fb7b31e","Type":"ContainerDied","Data":"9113dbab499819be8ff46510eecd467629a86caa54f81930ac138d55e1e55774"}
Oct 30 22:07:02.277070 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:07:02.277027    4412 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-7476d02544/isvc-pmml-v2-kserve-predictor-ff9978f97-p7f27" podUID="3e9d15e2-b8d8-45f5-98c2-9b112fb7b31e" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.20:8643/healthz\": dial tcp 10.133.0.20:8643: connect: connection refused"
Oct 30 22:07:02.380264 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:07:02.380241    4412 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-7476d02544/isvc-pmml-v2-kserve-predictor-ff9978f97-p7f27"
Oct 30 22:07:02.479257 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:07:02.479221    4412 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3e9d15e2-b8d8-45f5-98c2-9b112fb7b31e-kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/3e9d15e2-b8d8-45f5-98c2-9b112fb7b31e-kserve-provision-location\") pod \"3e9d15e2-b8d8-45f5-98c2-9b112fb7b31e\" (UID: \"3e9d15e2-b8d8-45f5-98c2-9b112fb7b31e\") "
Oct 30 22:07:02.479447 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:07:02.479293    4412 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/3e9d15e2-b8d8-45f5-98c2-9b112fb7b31e-proxy-tls\" (UniqueName: \"kubernetes.io/secret/3e9d15e2-b8d8-45f5-98c2-9b112fb7b31e-proxy-tls\") pod \"3e9d15e2-b8d8-45f5-98c2-9b112fb7b31e\" (UID: \"3e9d15e2-b8d8-45f5-98c2-9b112fb7b31e\") "
Oct 30 22:07:02.479447 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:07:02.479331    4412 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3e9d15e2-b8d8-45f5-98c2-9b112fb7b31e-kube-api-access-smpzz\" (UniqueName: \"kubernetes.io/projected/3e9d15e2-b8d8-45f5-98c2-9b112fb7b31e-kube-api-access-smpzz\") pod \"3e9d15e2-b8d8-45f5-98c2-9b112fb7b31e\" (UID: \"3e9d15e2-b8d8-45f5-98c2-9b112fb7b31e\") "
Oct 30 22:07:02.479447 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:07:02.479387    4412 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/3e9d15e2-b8d8-45f5-98c2-9b112fb7b31e-kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/3e9d15e2-b8d8-45f5-98c2-9b112fb7b31e-kube-rbac-proxy-sar-config\") pod \"3e9d15e2-b8d8-45f5-98c2-9b112fb7b31e\" (UID: \"3e9d15e2-b8d8-45f5-98c2-9b112fb7b31e\") "
Oct 30 22:07:02.479447 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:07:02.479422    4412 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/3e9d15e2-b8d8-45f5-98c2-9b112fb7b31e-cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/3e9d15e2-b8d8-45f5-98c2-9b112fb7b31e-cabundle-cert\") pod \"3e9d15e2-b8d8-45f5-98c2-9b112fb7b31e\" (UID: \"3e9d15e2-b8d8-45f5-98c2-9b112fb7b31e\") "
Oct 30 22:07:02.479660 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:07:02.479572    4412 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3e9d15e2-b8d8-45f5-98c2-9b112fb7b31e-kserve-provision-location" pod "3e9d15e2-b8d8-45f5-98c2-9b112fb7b31e" (UID: "3e9d15e2-b8d8-45f5-98c2-9b112fb7b31e"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:07:02.479777 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:07:02.479747    4412 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3e9d15e2-b8d8-45f5-98c2-9b112fb7b31e-kube-rbac-proxy-sar-config" pod "3e9d15e2-b8d8-45f5-98c2-9b112fb7b31e" (UID: "3e9d15e2-b8d8-45f5-98c2-9b112fb7b31e"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:07:02.479856 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:07:02.479790    4412 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3e9d15e2-b8d8-45f5-98c2-9b112fb7b31e-cabundle-cert" pod "3e9d15e2-b8d8-45f5-98c2-9b112fb7b31e" (UID: "3e9d15e2-b8d8-45f5-98c2-9b112fb7b31e"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:07:02.481697 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:07:02.481676    4412 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3e9d15e2-b8d8-45f5-98c2-9b112fb7b31e-kube-api-access-smpzz" pod "3e9d15e2-b8d8-45f5-98c2-9b112fb7b31e" (UID: "3e9d15e2-b8d8-45f5-98c2-9b112fb7b31e"). InnerVolumeSpecName "kube-api-access-smpzz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:07:02.481805 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:07:02.481706    4412 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3e9d15e2-b8d8-45f5-98c2-9b112fb7b31e-proxy-tls" pod "3e9d15e2-b8d8-45f5-98c2-9b112fb7b31e" (UID: "3e9d15e2-b8d8-45f5-98c2-9b112fb7b31e"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:07:02.580193 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:07:02.580109    4412 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/3e9d15e2-b8d8-45f5-98c2-9b112fb7b31e-proxy-tls\") on node \"ip-10-0-131-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:07:02.580193 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:07:02.580146    4412 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-smpzz\" (UniqueName: \"kubernetes.io/projected/3e9d15e2-b8d8-45f5-98c2-9b112fb7b31e-kube-api-access-smpzz\") on node \"ip-10-0-131-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:07:02.580193 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:07:02.580157    4412 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/3e9d15e2-b8d8-45f5-98c2-9b112fb7b31e-kube-rbac-proxy-sar-config\") on node \"ip-10-0-131-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:07:02.580193 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:07:02.580168    4412 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/3e9d15e2-b8d8-45f5-98c2-9b112fb7b31e-cabundle-cert\") on node \"ip-10-0-131-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:07:02.580193 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:07:02.580182    4412 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/3e9d15e2-b8d8-45f5-98c2-9b112fb7b31e-kserve-provision-location\") on node \"ip-10-0-131-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:07:02.666085 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:07:02.666053    4412 generic.go:356] "Generic (PLEG): container finished" podID="3e9d15e2-b8d8-45f5-98c2-9b112fb7b31e" containerID="3ca54bbb04c1327a7698d972263c71372ad36cc3e12dbe8521239b7402777c81" exitCode=0
Oct 30 22:07:02.666267 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:07:02.666103    4412 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-7476d02544/isvc-pmml-v2-kserve-predictor-ff9978f97-p7f27" event={"ID":"3e9d15e2-b8d8-45f5-98c2-9b112fb7b31e","Type":"ContainerDied","Data":"3ca54bbb04c1327a7698d972263c71372ad36cc3e12dbe8521239b7402777c81"}
Oct 30 22:07:02.666267 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:07:02.666152    4412 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-7476d02544/isvc-pmml-v2-kserve-predictor-ff9978f97-p7f27"
Oct 30 22:07:02.666267 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:07:02.666176    4412 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-7476d02544/isvc-pmml-v2-kserve-predictor-ff9978f97-p7f27" event={"ID":"3e9d15e2-b8d8-45f5-98c2-9b112fb7b31e","Type":"ContainerDied","Data":"3f67b6ab72ae9bc77b772bd0fa1a0b7972a54f4c099370959674e622cf89d51d"}
Oct 30 22:07:02.666267 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:07:02.666205    4412 scope.go:122] "RemoveContainer" containerID="9113dbab499819be8ff46510eecd467629a86caa54f81930ac138d55e1e55774"
Oct 30 22:07:02.674835 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:07:02.674817    4412 scope.go:122] "RemoveContainer" containerID="3ca54bbb04c1327a7698d972263c71372ad36cc3e12dbe8521239b7402777c81"
Oct 30 22:07:02.682205 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:07:02.682187    4412 scope.go:122] "RemoveContainer" containerID="657df0d2d692f93f21897b694f6ef14c5e067a9a5cbae784523bf0ee972c4f8a"
Oct 30 22:07:02.689218 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:07:02.689203    4412 scope.go:122] "RemoveContainer" containerID="9113dbab499819be8ff46510eecd467629a86caa54f81930ac138d55e1e55774"
Oct 30 22:07:02.689473 ip-10-0-131-202 kubenswrapper[4412]: E1030 22:07:02.689453    4412 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9113dbab499819be8ff46510eecd467629a86caa54f81930ac138d55e1e55774\": container with ID starting with 9113dbab499819be8ff46510eecd467629a86caa54f81930ac138d55e1e55774 not found: ID does not exist" containerID="9113dbab499819be8ff46510eecd467629a86caa54f81930ac138d55e1e55774"
Oct 30 22:07:02.689520 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:07:02.689482    4412 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9113dbab499819be8ff46510eecd467629a86caa54f81930ac138d55e1e55774"} err="failed to get container status \"9113dbab499819be8ff46510eecd467629a86caa54f81930ac138d55e1e55774\": rpc error: code = NotFound desc = could not find container \"9113dbab499819be8ff46510eecd467629a86caa54f81930ac138d55e1e55774\": container with ID starting with 9113dbab499819be8ff46510eecd467629a86caa54f81930ac138d55e1e55774 not found: ID does not exist"
Oct 30 22:07:02.689520 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:07:02.689500    4412 scope.go:122] "RemoveContainer" containerID="3ca54bbb04c1327a7698d972263c71372ad36cc3e12dbe8521239b7402777c81"
Oct 30 22:07:02.689719 ip-10-0-131-202 kubenswrapper[4412]: E1030 22:07:02.689700    4412 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3ca54bbb04c1327a7698d972263c71372ad36cc3e12dbe8521239b7402777c81\": container with ID starting with 3ca54bbb04c1327a7698d972263c71372ad36cc3e12dbe8521239b7402777c81 not found: ID does not exist" containerID="3ca54bbb04c1327a7698d972263c71372ad36cc3e12dbe8521239b7402777c81"
Oct 30 22:07:02.689757 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:07:02.689727    4412 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3ca54bbb04c1327a7698d972263c71372ad36cc3e12dbe8521239b7402777c81"} err="failed to get container status \"3ca54bbb04c1327a7698d972263c71372ad36cc3e12dbe8521239b7402777c81\": rpc error: code = NotFound desc = could not find container \"3ca54bbb04c1327a7698d972263c71372ad36cc3e12dbe8521239b7402777c81\": container with ID starting with 3ca54bbb04c1327a7698d972263c71372ad36cc3e12dbe8521239b7402777c81 not found: ID does not exist"
Oct 30 22:07:02.689757 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:07:02.689743    4412 scope.go:122] "RemoveContainer" containerID="657df0d2d692f93f21897b694f6ef14c5e067a9a5cbae784523bf0ee972c4f8a"
Oct 30 22:07:02.689985 ip-10-0-131-202 kubenswrapper[4412]: E1030 22:07:02.689967    4412 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"657df0d2d692f93f21897b694f6ef14c5e067a9a5cbae784523bf0ee972c4f8a\": container with ID starting with 657df0d2d692f93f21897b694f6ef14c5e067a9a5cbae784523bf0ee972c4f8a not found: ID does not exist" containerID="657df0d2d692f93f21897b694f6ef14c5e067a9a5cbae784523bf0ee972c4f8a"
Oct 30 22:07:02.690019 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:07:02.689991    4412 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"657df0d2d692f93f21897b694f6ef14c5e067a9a5cbae784523bf0ee972c4f8a"} err="failed to get container status \"657df0d2d692f93f21897b694f6ef14c5e067a9a5cbae784523bf0ee972c4f8a\": rpc error: code = NotFound desc = could not find container \"657df0d2d692f93f21897b694f6ef14c5e067a9a5cbae784523bf0ee972c4f8a\": container with ID starting with 657df0d2d692f93f21897b694f6ef14c5e067a9a5cbae784523bf0ee972c4f8a not found: ID does not exist"
Oct 30 22:07:02.832273 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:07:02.832190    4412 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw1-7476d02544/isvc-pmml-v2-kserve-predictor-ff9978f97-p7f27"]
Oct 30 22:07:02.835626 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:07:02.835600    4412 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-gw1-7476d02544/isvc-pmml-v2-kserve-predictor-ff9978f97-p7f27"]
Oct 30 22:07:03.626644 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:07:03.626615    4412 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="3e9d15e2-b8d8-45f5-98c2-9b112fb7b31e" path="/var/lib/kubelet/pods/3e9d15e2-b8d8-45f5-98c2-9b112fb7b31e/volumes"
Oct 30 22:07:04.448908 ip-10-0-131-202 kubenswrapper[4412]: E1030 22:07:04.448868    4412 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bkhmz" containerName="extractor"
Oct 30 22:07:34.448662 ip-10-0-131-202 kubenswrapper[4412]: E1030 22:07:34.448584    4412 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bkhmz" containerName="extractor"
Oct 30 22:08:04.448256 ip-10-0-131-202 kubenswrapper[4412]: E1030 22:08:04.448217    4412 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bkhmz" containerName="extractor"
Oct 30 22:08:34.448470 ip-10-0-131-202 kubenswrapper[4412]: E1030 22:08:34.448435    4412 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bkhmz" containerName="extractor"
Oct 30 22:09:04.448329 ip-10-0-131-202 kubenswrapper[4412]: E1030 22:09:04.448290    4412 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bkhmz" containerName="extractor"
Oct 30 22:09:34.448068 ip-10-0-131-202 kubenswrapper[4412]: E1030 22:09:34.448020    4412 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bkhmz" containerName="extractor"
Oct 30 22:10:04.448168 ip-10-0-131-202 kubenswrapper[4412]: E1030 22:10:04.448105    4412 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bkhmz" containerName="extractor"
Oct 30 22:10:34.448606 ip-10-0-131-202 kubenswrapper[4412]: E1030 22:10:34.448519    4412 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bkhmz" containerName="extractor"
Oct 30 22:11:04.447847 ip-10-0-131-202 kubenswrapper[4412]: E1030 22:11:04.447801    4412 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bkhmz" containerName="extractor"
Oct 30 22:11:34.448639 ip-10-0-131-202 kubenswrapper[4412]: E1030 22:11:34.448605    4412 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bkhmz" containerName="extractor"
Oct 30 22:12:04.448245 ip-10-0-131-202 kubenswrapper[4412]: E1030 22:12:04.448213    4412 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bkhmz" containerName="extractor"
Oct 30 22:12:34.448516 ip-10-0-131-202 kubenswrapper[4412]: E1030 22:12:34.448479    4412 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bkhmz" containerName="extractor"
Oct 30 22:13:04.448188 ip-10-0-131-202 kubenswrapper[4412]: E1030 22:13:04.448151    4412 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bkhmz" containerName="extractor"
Oct 30 22:13:34.448050 ip-10-0-131-202 kubenswrapper[4412]: E1030 22:13:34.447960    4412 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bkhmz" containerName="extractor"
Oct 30 22:14:04.448790 ip-10-0-131-202 kubenswrapper[4412]: E1030 22:14:04.448749    4412 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bkhmz" containerName="extractor"
Oct 30 22:14:34.447922 ip-10-0-131-202 kubenswrapper[4412]: E1030 22:14:34.447887    4412 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bkhmz" containerName="extractor"
Oct 30 22:15:04.447588 ip-10-0-131-202 kubenswrapper[4412]: E1030 22:15:04.447552    4412 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bkhmz" containerName="extractor"
Oct 30 22:15:34.448061 ip-10-0-131-202 kubenswrapper[4412]: E1030 22:15:34.448010    4412 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bkhmz" containerName="extractor"
Oct 30 22:16:04.447917 ip-10-0-131-202 kubenswrapper[4412]: E1030 22:16:04.447877    4412 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bkhmz" containerName="extractor"
Oct 30 22:16:34.448141 ip-10-0-131-202 kubenswrapper[4412]: E1030 22:16:34.448034    4412 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bkhmz" containerName="extractor"
Oct 30 22:17:04.448435 ip-10-0-131-202 kubenswrapper[4412]: E1030 22:17:04.448399    4412 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bkhmz" containerName="extractor"
Oct 30 22:17:28.645383 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:17:28.645347    4412 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kserve/curl-metrics-fa7ecc11"]
Oct 30 22:17:28.645796 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:17:28.645615    4412 cpu_manager.go:414] "containerMap: removing container" podUID="3e9d15e2-b8d8-45f5-98c2-9b112fb7b31e" containerName="kserve-container"
Oct 30 22:17:28.645796 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:17:28.645626    4412 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3e9d15e2-b8d8-45f5-98c2-9b112fb7b31e" containerName="kserve-container"
Oct 30 22:17:28.645796 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:17:28.645642    4412 cpu_manager.go:414] "containerMap: removing container" podUID="3e9d15e2-b8d8-45f5-98c2-9b112fb7b31e" containerName="storage-initializer"
Oct 30 22:17:28.645796 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:17:28.645648    4412 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3e9d15e2-b8d8-45f5-98c2-9b112fb7b31e" containerName="storage-initializer"
Oct 30 22:17:28.645796 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:17:28.645658    4412 cpu_manager.go:414] "containerMap: removing container" podUID="3e9d15e2-b8d8-45f5-98c2-9b112fb7b31e" containerName="kube-rbac-proxy"
Oct 30 22:17:28.645796 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:17:28.645662    4412 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3e9d15e2-b8d8-45f5-98c2-9b112fb7b31e" containerName="kube-rbac-proxy"
Oct 30 22:17:28.645796 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:17:28.645715    4412 memory_manager.go:365] "RemoveStaleState removing state" podUID="3e9d15e2-b8d8-45f5-98c2-9b112fb7b31e" containerName="kserve-container"
Oct 30 22:17:28.645796 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:17:28.645724    4412 memory_manager.go:365] "RemoveStaleState removing state" podUID="3e9d15e2-b8d8-45f5-98c2-9b112fb7b31e" containerName="kube-rbac-proxy"
Oct 30 22:17:28.648532 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:17:28.648516    4412 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/curl-metrics-fa7ecc11"
Oct 30 22:17:28.669741 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:17:28.669706    4412 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cs264\" (UniqueName: \"kubernetes.io/projected/3a85499e-445a-404a-ba9e-2f83a393687d-kube-api-access-cs264\") pod \"curl-metrics-fa7ecc11\" (UID: \"3a85499e-445a-404a-ba9e-2f83a393687d\") " pod="kserve/curl-metrics-fa7ecc11"
Oct 30 22:17:28.705358 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:17:28.705328    4412 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kserve/curl-metrics-fa7ecc11"]
Oct 30 22:17:28.754508 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:17:28.754484    4412 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"kube-root-ca.crt\""
Oct 30 22:17:28.754508 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:17:28.754484    4412 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"default-dockercfg-p47c2\""
Oct 30 22:17:28.754689 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:17:28.754530    4412 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"openshift-service-ca.crt\""
Oct 30 22:17:28.770934 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:17:28.770912    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cs264\" (UniqueName: \"kubernetes.io/projected/3a85499e-445a-404a-ba9e-2f83a393687d-kube-api-access-cs264\") pod \"curl-metrics-fa7ecc11\" (UID: \"3a85499e-445a-404a-ba9e-2f83a393687d\") " pod="kserve/curl-metrics-fa7ecc11"
Oct 30 22:17:28.828627 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:17:28.828586    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cs264\" (UniqueName: \"kubernetes.io/projected/3a85499e-445a-404a-ba9e-2f83a393687d-kube-api-access-cs264\") pod \"curl-metrics-fa7ecc11\" (UID: \"3a85499e-445a-404a-ba9e-2f83a393687d\") " pod="kserve/curl-metrics-fa7ecc11"
Oct 30 22:17:28.972358 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:17:28.972336    4412 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/curl-metrics-fa7ecc11"
Oct 30 22:17:29.338002 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:17:29.337921    4412 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kserve/curl-metrics-fa7ecc11"]
Oct 30 22:17:29.342454 ip-10-0-131-202 kubenswrapper[4412]: W1030 22:17:29.342426    4412 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3a85499e_445a_404a_ba9e_2f83a393687d.slice/crio-4975d983fb9dacd81c9ded00150de042c492bbb8f314a0edcfb866bdd9e239dd WatchSource:0}: Error finding container 4975d983fb9dacd81c9ded00150de042c492bbb8f314a0edcfb866bdd9e239dd: Status 404 returned error can't find the container with id 4975d983fb9dacd81c9ded00150de042c492bbb8f314a0edcfb866bdd9e239dd
Oct 30 22:17:29.345039 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:17:29.345025    4412 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 22:17:29.888720 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:17:29.888687    4412 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kserve/curl-metrics-fa7ecc11" event={"ID":"3a85499e-445a-404a-ba9e-2f83a393687d","Type":"ContainerStarted","Data":"4975d983fb9dacd81c9ded00150de042c492bbb8f314a0edcfb866bdd9e239dd"}
Oct 30 22:17:31.897082 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:17:31.897047    4412 generic.go:356] "Generic (PLEG): container finished" podID="3a85499e-445a-404a-ba9e-2f83a393687d" containerID="d97172cfd8f14db6c45199c95549ce8e946ca30dbd2e8dc74202e48dd14027a3" exitCode=0
Oct 30 22:17:31.897488 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:17:31.897144    4412 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kserve/curl-metrics-fa7ecc11" event={"ID":"3a85499e-445a-404a-ba9e-2f83a393687d","Type":"ContainerDied","Data":"d97172cfd8f14db6c45199c95549ce8e946ca30dbd2e8dc74202e48dd14027a3"}
Oct 30 22:17:33.025708 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:17:33.025685    4412 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve/curl-metrics-fa7ecc11"
Oct 30 22:17:33.102742 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:17:33.102719    4412 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3a85499e-445a-404a-ba9e-2f83a393687d-kube-api-access-cs264\" (UniqueName: \"kubernetes.io/projected/3a85499e-445a-404a-ba9e-2f83a393687d-kube-api-access-cs264\") pod \"3a85499e-445a-404a-ba9e-2f83a393687d\" (UID: \"3a85499e-445a-404a-ba9e-2f83a393687d\") "
Oct 30 22:17:33.104787 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:17:33.104765    4412 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3a85499e-445a-404a-ba9e-2f83a393687d-kube-api-access-cs264" pod "3a85499e-445a-404a-ba9e-2f83a393687d" (UID: "3a85499e-445a-404a-ba9e-2f83a393687d"). InnerVolumeSpecName "kube-api-access-cs264". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:17:33.203400 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:17:33.203342    4412 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cs264\" (UniqueName: \"kubernetes.io/projected/3a85499e-445a-404a-ba9e-2f83a393687d-kube-api-access-cs264\") on node \"ip-10-0-131-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:17:33.319325 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:17:33.319295    4412 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_curl-metrics-fa7ecc11_3a85499e-445a-404a-ba9e-2f83a393687d/curl-metrics-fa7ecc11/0.log"
Oct 30 22:17:33.434231 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:17:33.434205    4412 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kserve/curl-metrics-fa7ecc11"]
Oct 30 22:17:33.437788 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:17:33.437761    4412 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["kserve/curl-metrics-fa7ecc11"]
Oct 30 22:17:33.625776 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:17:33.625749    4412 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="3a85499e-445a-404a-ba9e-2f83a393687d" path="/var/lib/kubelet/pods/3a85499e-445a-404a-ba9e-2f83a393687d/volumes"
Oct 30 22:17:33.906380 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:17:33.906303    4412 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve/curl-metrics-fa7ecc11"
Oct 30 22:17:33.906380 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:17:33.906327    4412 scope.go:122] "RemoveContainer" containerID="d97172cfd8f14db6c45199c95549ce8e946ca30dbd2e8dc74202e48dd14027a3"
Oct 30 22:17:34.009642 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:17:34.009617    4412 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kserve/curl-metrics-4814f50e"]
Oct 30 22:17:34.009945 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:17:34.009932    4412 cpu_manager.go:414] "containerMap: removing container" podUID="3a85499e-445a-404a-ba9e-2f83a393687d" containerName="curl-metrics-fa7ecc11"
Oct 30 22:17:34.009987 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:17:34.009946    4412 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3a85499e-445a-404a-ba9e-2f83a393687d" containerName="curl-metrics-fa7ecc11"
Oct 30 22:17:34.010021 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:17:34.010006    4412 memory_manager.go:365] "RemoveStaleState removing state" podUID="3a85499e-445a-404a-ba9e-2f83a393687d" containerName="curl-metrics-fa7ecc11"
Oct 30 22:17:34.014020 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:17:34.014005    4412 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/curl-metrics-4814f50e"
Oct 30 22:17:34.068161 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:17:34.068139    4412 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"default-dockercfg-p47c2\""
Oct 30 22:17:34.069563 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:17:34.069542    4412 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kserve/curl-metrics-4814f50e"]
Oct 30 22:17:34.109314 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:17:34.109293    4412 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-92g4g\" (UniqueName: \"kubernetes.io/projected/d3eecf87-12b7-45d4-b70c-b0c683c2aebe-kube-api-access-92g4g\") pod \"curl-metrics-4814f50e\" (UID: \"d3eecf87-12b7-45d4-b70c-b0c683c2aebe\") " pod="kserve/curl-metrics-4814f50e"
Oct 30 22:17:34.118465 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:17:34.118450    4412 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"openshift-service-ca.crt\""
Oct 30 22:17:34.118538 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:17:34.118469    4412 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"kube-root-ca.crt\""
Oct 30 22:17:34.209908 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:17:34.209854    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-92g4g\" (UniqueName: \"kubernetes.io/projected/d3eecf87-12b7-45d4-b70c-b0c683c2aebe-kube-api-access-92g4g\") pod \"curl-metrics-4814f50e\" (UID: \"d3eecf87-12b7-45d4-b70c-b0c683c2aebe\") " pod="kserve/curl-metrics-4814f50e"
Oct 30 22:17:34.267441 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:17:34.267420    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-92g4g\" (UniqueName: \"kubernetes.io/projected/d3eecf87-12b7-45d4-b70c-b0c683c2aebe-kube-api-access-92g4g\") pod \"curl-metrics-4814f50e\" (UID: \"d3eecf87-12b7-45d4-b70c-b0c683c2aebe\") " pod="kserve/curl-metrics-4814f50e"
Oct 30 22:17:34.334687 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:17:34.334667    4412 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/curl-metrics-4814f50e"
Oct 30 22:17:34.447643 ip-10-0-131-202 kubenswrapper[4412]: E1030 22:17:34.447610    4412 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bkhmz" containerName="extractor"
Oct 30 22:17:34.700553 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:17:34.700533    4412 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kserve/curl-metrics-4814f50e"]
Oct 30 22:17:34.703098 ip-10-0-131-202 kubenswrapper[4412]: W1030 22:17:34.703072    4412 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd3eecf87_12b7_45d4_b70c_b0c683c2aebe.slice/crio-34ae27bcc0c749df803284121a04f4d473bbd7c34304c3e24360d9e3ff669ed2 WatchSource:0}: Error finding container 34ae27bcc0c749df803284121a04f4d473bbd7c34304c3e24360d9e3ff669ed2: Status 404 returned error can't find the container with id 34ae27bcc0c749df803284121a04f4d473bbd7c34304c3e24360d9e3ff669ed2
Oct 30 22:17:34.910857 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:17:34.910828    4412 generic.go:356] "Generic (PLEG): container finished" podID="d3eecf87-12b7-45d4-b70c-b0c683c2aebe" containerID="0761a837c8861b42a464c649b1418ec94e75d27f512ba82e4aa6e63c376e8351" exitCode=0
Oct 30 22:17:34.910993 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:17:34.910910    4412 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kserve/curl-metrics-4814f50e" event={"ID":"d3eecf87-12b7-45d4-b70c-b0c683c2aebe","Type":"ContainerDied","Data":"0761a837c8861b42a464c649b1418ec94e75d27f512ba82e4aa6e63c376e8351"}
Oct 30 22:17:34.910993 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:17:34.910953    4412 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kserve/curl-metrics-4814f50e" event={"ID":"d3eecf87-12b7-45d4-b70c-b0c683c2aebe","Type":"ContainerStarted","Data":"34ae27bcc0c749df803284121a04f4d473bbd7c34304c3e24360d9e3ff669ed2"}
Oct 30 22:17:36.031502 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:17:36.031481    4412 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve/curl-metrics-4814f50e"
Oct 30 22:17:36.124649 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:17:36.124627    4412 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d3eecf87-12b7-45d4-b70c-b0c683c2aebe-kube-api-access-92g4g\" (UniqueName: \"kubernetes.io/projected/d3eecf87-12b7-45d4-b70c-b0c683c2aebe-kube-api-access-92g4g\") pod \"d3eecf87-12b7-45d4-b70c-b0c683c2aebe\" (UID: \"d3eecf87-12b7-45d4-b70c-b0c683c2aebe\") "
Oct 30 22:17:36.126785 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:17:36.126759    4412 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d3eecf87-12b7-45d4-b70c-b0c683c2aebe-kube-api-access-92g4g" pod "d3eecf87-12b7-45d4-b70c-b0c683c2aebe" (UID: "d3eecf87-12b7-45d4-b70c-b0c683c2aebe"). InnerVolumeSpecName "kube-api-access-92g4g". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:17:36.173523 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:17:36.173469    4412 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_curl-metrics-4814f50e_d3eecf87-12b7-45d4-b70c-b0c683c2aebe/curl-metrics-4814f50e/0.log"
Oct 30 22:17:36.225682 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:17:36.225658    4412 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-92g4g\" (UniqueName: \"kubernetes.io/projected/d3eecf87-12b7-45d4-b70c-b0c683c2aebe-kube-api-access-92g4g\") on node \"ip-10-0-131-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:17:36.298950 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:17:36.298927    4412 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kserve/curl-metrics-4814f50e"]
Oct 30 22:17:36.302641 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:17:36.302625    4412 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["kserve/curl-metrics-4814f50e"]
Oct 30 22:17:36.920065 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:17:36.920034    4412 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve/curl-metrics-4814f50e"
Oct 30 22:17:36.920065 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:17:36.920051    4412 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="34ae27bcc0c749df803284121a04f4d473bbd7c34304c3e24360d9e3ff669ed2"
Oct 30 22:17:36.938400 ip-10-0-131-202 kubenswrapper[4412]: E1030 22:17:36.938377    4412 cadvisor_stats_provider.go:569] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd3eecf87_12b7_45d4_b70c_b0c683c2aebe.slice\": RecentStats: unable to find data in memory cache]"
Oct 30 22:17:36.938520 ip-10-0-131-202 kubenswrapper[4412]: E1030 22:17:36.938483    4412 cadvisor_stats_provider.go:569] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd3eecf87_12b7_45d4_b70c_b0c683c2aebe.slice\": RecentStats: unable to find data in memory cache]"
Oct 30 22:17:36.981269 ip-10-0-131-202 kubenswrapper[4412]: E1030 22:17:36.981241    4412 status_manager.go:1045] "Failed to get status for pod" err="pods \"curl-metrics-4814f50e\" is forbidden: User \"system:node:ip-10-0-131-202.us-west-1.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kserve\": no relationship found between node 'ip-10-0-131-202.us-west-1.compute.internal' and this object" podUID="d3eecf87-12b7-45d4-b70c-b0c683c2aebe" pod="kserve/curl-metrics-4814f50e"
Oct 30 22:17:37.626076 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:17:37.626046    4412 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d3eecf87-12b7-45d4-b70c-b0c683c2aebe" path="/var/lib/kubelet/pods/d3eecf87-12b7-45d4-b70c-b0c683c2aebe/volumes"
Oct 30 22:18:04.448461 ip-10-0-131-202 kubenswrapper[4412]: E1030 22:18:04.448427    4412 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bkhmz" containerName="extractor"
Oct 30 22:18:34.448442 ip-10-0-131-202 kubenswrapper[4412]: E1030 22:18:34.448407    4412 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bkhmz" containerName="extractor"
Oct 30 22:19:04.447905 ip-10-0-131-202 kubenswrapper[4412]: E1030 22:19:04.447863    4412 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bkhmz" containerName="extractor"
Oct 30 22:19:34.447506 ip-10-0-131-202 kubenswrapper[4412]: E1030 22:19:34.447431    4412 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bkhmz" containerName="extractor"
Oct 30 22:20:04.447832 ip-10-0-131-202 kubenswrapper[4412]: E1030 22:20:04.447785    4412 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bkhmz" containerName="extractor"
Oct 30 22:20:34.447617 ip-10-0-131-202 kubenswrapper[4412]: E1030 22:20:34.447567    4412 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bkhmz" containerName="extractor"
Oct 30 22:21:04.447572 ip-10-0-131-202 kubenswrapper[4412]: E1030 22:21:04.447530    4412 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bkhmz" containerName="extractor"
Oct 30 22:21:34.448196 ip-10-0-131-202 kubenswrapper[4412]: E1030 22:21:34.448154    4412 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bkhmz" containerName="extractor"
Oct 30 22:22:04.448252 ip-10-0-131-202 kubenswrapper[4412]: E1030 22:22:04.448212    4412 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bkhmz" containerName="extractor"
Oct 30 22:22:34.448201 ip-10-0-131-202 kubenswrapper[4412]: E1030 22:22:34.448165    4412 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bkhmz" containerName="extractor"
Oct 30 22:23:04.448305 ip-10-0-131-202 kubenswrapper[4412]: E1030 22:23:04.448263    4412 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bkhmz" containerName="extractor"
Oct 30 22:23:34.448839 ip-10-0-131-202 kubenswrapper[4412]: E1030 22:23:34.448795    4412 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bkhmz" containerName="extractor"
Oct 30 22:23:36.170365 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:23:36.170332    4412 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-gw0-37d062d1dd/isvc-autogluon-v1-predictor-7c64bdbd96-gjwcx"]
Oct 30 22:23:36.170757 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:23:36.170616    4412 cpu_manager.go:414] "containerMap: removing container" podUID="d3eecf87-12b7-45d4-b70c-b0c683c2aebe" containerName="curl-metrics-4814f50e"
Oct 30 22:23:36.170757 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:23:36.170627    4412 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d3eecf87-12b7-45d4-b70c-b0c683c2aebe" containerName="curl-metrics-4814f50e"
Oct 30 22:23:36.170757 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:23:36.170679    4412 memory_manager.go:365] "RemoveStaleState removing state" podUID="d3eecf87-12b7-45d4-b70c-b0c683c2aebe" containerName="curl-metrics-4814f50e"
Oct 30 22:23:36.173688 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:23:36.173670    4412 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-37d062d1dd/isvc-autogluon-v1-predictor-7c64bdbd96-gjwcx"
Oct 30 22:23:36.194262 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:23:36.194232    4412 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/8385df9b-f397-457b-b0d7-11c49429c858-kserve-provision-location\") pod \"isvc-autogluon-v1-predictor-7c64bdbd96-gjwcx\" (UID: \"8385df9b-f397-457b-b0d7-11c49429c858\") " pod="e2e-gw0-37d062d1dd/isvc-autogluon-v1-predictor-7c64bdbd96-gjwcx"
Oct 30 22:23:36.194397 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:23:36.194266    4412 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/8385df9b-f397-457b-b0d7-11c49429c858-proxy-tls\") pod \"isvc-autogluon-v1-predictor-7c64bdbd96-gjwcx\" (UID: \"8385df9b-f397-457b-b0d7-11c49429c858\") " pod="e2e-gw0-37d062d1dd/isvc-autogluon-v1-predictor-7c64bdbd96-gjwcx"
Oct 30 22:23:36.194397 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:23:36.194302    4412 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h8sxw\" (UniqueName: \"kubernetes.io/projected/8385df9b-f397-457b-b0d7-11c49429c858-kube-api-access-h8sxw\") pod \"isvc-autogluon-v1-predictor-7c64bdbd96-gjwcx\" (UID: \"8385df9b-f397-457b-b0d7-11c49429c858\") " pod="e2e-gw0-37d062d1dd/isvc-autogluon-v1-predictor-7c64bdbd96-gjwcx"
Oct 30 22:23:36.194397 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:23:36.194390    4412 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/8385df9b-f397-457b-b0d7-11c49429c858-kube-rbac-proxy-sar-config\") pod \"isvc-autogluon-v1-predictor-7c64bdbd96-gjwcx\" (UID: \"8385df9b-f397-457b-b0d7-11c49429c858\") " pod="e2e-gw0-37d062d1dd/isvc-autogluon-v1-predictor-7c64bdbd96-gjwcx"
Oct 30 22:23:36.194516 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:23:36.194412    4412 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/8385df9b-f397-457b-b0d7-11c49429c858-cabundle-cert\") pod \"isvc-autogluon-v1-predictor-7c64bdbd96-gjwcx\" (UID: \"8385df9b-f397-457b-b0d7-11c49429c858\") " pod="e2e-gw0-37d062d1dd/isvc-autogluon-v1-predictor-7c64bdbd96-gjwcx"
Oct 30 22:23:36.227220 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:23:36.227195    4412 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw0-37d062d1dd\"/\"seaweedfs-s3-creds\""
Oct 30 22:23:36.277854 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:23:36.277831    4412 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw0-37d062d1dd\"/\"odh-kserve-custom-ca-bundle\""
Oct 30 22:23:36.295385 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:23:36.295351    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/8385df9b-f397-457b-b0d7-11c49429c858-kserve-provision-location\") pod \"isvc-autogluon-v1-predictor-7c64bdbd96-gjwcx\" (UID: \"8385df9b-f397-457b-b0d7-11c49429c858\") " pod="e2e-gw0-37d062d1dd/isvc-autogluon-v1-predictor-7c64bdbd96-gjwcx"
Oct 30 22:23:36.295385 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:23:36.295387    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/8385df9b-f397-457b-b0d7-11c49429c858-proxy-tls\") pod \"isvc-autogluon-v1-predictor-7c64bdbd96-gjwcx\" (UID: \"8385df9b-f397-457b-b0d7-11c49429c858\") " pod="e2e-gw0-37d062d1dd/isvc-autogluon-v1-predictor-7c64bdbd96-gjwcx"
Oct 30 22:23:36.295572 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:23:36.295413    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h8sxw\" (UniqueName: \"kubernetes.io/projected/8385df9b-f397-457b-b0d7-11c49429c858-kube-api-access-h8sxw\") pod \"isvc-autogluon-v1-predictor-7c64bdbd96-gjwcx\" (UID: \"8385df9b-f397-457b-b0d7-11c49429c858\") " pod="e2e-gw0-37d062d1dd/isvc-autogluon-v1-predictor-7c64bdbd96-gjwcx"
Oct 30 22:23:36.295572 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:23:36.295447    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/8385df9b-f397-457b-b0d7-11c49429c858-kube-rbac-proxy-sar-config\") pod \"isvc-autogluon-v1-predictor-7c64bdbd96-gjwcx\" (UID: \"8385df9b-f397-457b-b0d7-11c49429c858\") " pod="e2e-gw0-37d062d1dd/isvc-autogluon-v1-predictor-7c64bdbd96-gjwcx"
Oct 30 22:23:36.295572 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:23:36.295465    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/8385df9b-f397-457b-b0d7-11c49429c858-cabundle-cert\") pod \"isvc-autogluon-v1-predictor-7c64bdbd96-gjwcx\" (UID: \"8385df9b-f397-457b-b0d7-11c49429c858\") " pod="e2e-gw0-37d062d1dd/isvc-autogluon-v1-predictor-7c64bdbd96-gjwcx"
Oct 30 22:23:36.295739 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:23:36.295720    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/8385df9b-f397-457b-b0d7-11c49429c858-kserve-provision-location\") pod \"isvc-autogluon-v1-predictor-7c64bdbd96-gjwcx\" (UID: \"8385df9b-f397-457b-b0d7-11c49429c858\") " pod="e2e-gw0-37d062d1dd/isvc-autogluon-v1-predictor-7c64bdbd96-gjwcx"
Oct 30 22:23:36.296089 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:23:36.296073    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/8385df9b-f397-457b-b0d7-11c49429c858-cabundle-cert\") pod \"isvc-autogluon-v1-predictor-7c64bdbd96-gjwcx\" (UID: \"8385df9b-f397-457b-b0d7-11c49429c858\") " pod="e2e-gw0-37d062d1dd/isvc-autogluon-v1-predictor-7c64bdbd96-gjwcx"
Oct 30 22:23:36.328224 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:23:36.328198    4412 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw0-37d062d1dd\"/\"default-dockercfg-4v4s8\""
Oct 30 22:23:36.328224 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:23:36.328222    4412 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw0-37d062d1dd\"/\"kube-root-ca.crt\""
Oct 30 22:23:36.328397 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:23:36.328199    4412 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw0-37d062d1dd\"/\"openshift-service-ca.crt\""
Oct 30 22:23:36.328397 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:23:36.328240    4412 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw0-37d062d1dd\"/\"isvc-autogluon-v1-predictor-serving-cert\""
Oct 30 22:23:36.328397 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:23:36.328258    4412 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw0-37d062d1dd\"/\"isvc-autogluon-v1-kube-rbac-proxy-sar-config\""
Oct 30 22:23:36.329528 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:23:36.329511    4412 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw0-37d062d1dd/isvc-autogluon-v1-predictor-7c64bdbd96-gjwcx"]
Oct 30 22:23:36.336899 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:23:36.336878    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/8385df9b-f397-457b-b0d7-11c49429c858-kube-rbac-proxy-sar-config\") pod \"isvc-autogluon-v1-predictor-7c64bdbd96-gjwcx\" (UID: \"8385df9b-f397-457b-b0d7-11c49429c858\") " pod="e2e-gw0-37d062d1dd/isvc-autogluon-v1-predictor-7c64bdbd96-gjwcx"
Oct 30 22:23:36.338795 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:23:36.338777    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/8385df9b-f397-457b-b0d7-11c49429c858-proxy-tls\") pod \"isvc-autogluon-v1-predictor-7c64bdbd96-gjwcx\" (UID: \"8385df9b-f397-457b-b0d7-11c49429c858\") " pod="e2e-gw0-37d062d1dd/isvc-autogluon-v1-predictor-7c64bdbd96-gjwcx"
Oct 30 22:23:36.353636 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:23:36.353610    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h8sxw\" (UniqueName: \"kubernetes.io/projected/8385df9b-f397-457b-b0d7-11c49429c858-kube-api-access-h8sxw\") pod \"isvc-autogluon-v1-predictor-7c64bdbd96-gjwcx\" (UID: \"8385df9b-f397-457b-b0d7-11c49429c858\") " pod="e2e-gw0-37d062d1dd/isvc-autogluon-v1-predictor-7c64bdbd96-gjwcx"
Oct 30 22:23:36.489887 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:23:36.489853    4412 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-37d062d1dd/isvc-autogluon-v1-predictor-7c64bdbd96-gjwcx"
Oct 30 22:23:36.910808 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:23:36.910659    4412 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw0-37d062d1dd/isvc-autogluon-v1-predictor-7c64bdbd96-gjwcx"]
Oct 30 22:23:36.913594 ip-10-0-131-202 kubenswrapper[4412]: W1030 22:23:36.913564    4412 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8385df9b_f397_457b_b0d7_11c49429c858.slice/crio-2b42d8f5360bf51d9764a98efea28263ec2af511b3835010a8049ff08a5f70c2 WatchSource:0}: Error finding container 2b42d8f5360bf51d9764a98efea28263ec2af511b3835010a8049ff08a5f70c2: Status 404 returned error can't find the container with id 2b42d8f5360bf51d9764a98efea28263ec2af511b3835010a8049ff08a5f70c2
Oct 30 22:23:37.196680 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:23:37.196593    4412 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-37d062d1dd/isvc-autogluon-v1-predictor-7c64bdbd96-gjwcx" event={"ID":"8385df9b-f397-457b-b0d7-11c49429c858","Type":"ContainerStarted","Data":"82dfac57dbb7a0cbdf86837804e2c3ebbfec1ad85b10e5c204cd2abc4d5045dd"}
Oct 30 22:23:37.197110 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:23:37.196697    4412 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-37d062d1dd/isvc-autogluon-v1-predictor-7c64bdbd96-gjwcx" event={"ID":"8385df9b-f397-457b-b0d7-11c49429c858","Type":"ContainerStarted","Data":"2b42d8f5360bf51d9764a98efea28263ec2af511b3835010a8049ff08a5f70c2"}
Oct 30 22:23:59.786809 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:23:59.786778    4412 scope.go:122] "RemoveContainer" containerID="0761a837c8861b42a464c649b1418ec94e75d27f512ba82e4aa6e63c376e8351"
Oct 30 22:24:04.447147 ip-10-0-131-202 kubenswrapper[4412]: E1030 22:24:04.447095    4412 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bkhmz" containerName="extractor"
Oct 30 22:24:18.346865 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:24:18.346782    4412 generic.go:356] "Generic (PLEG): container finished" podID="8385df9b-f397-457b-b0d7-11c49429c858" containerID="82dfac57dbb7a0cbdf86837804e2c3ebbfec1ad85b10e5c204cd2abc4d5045dd" exitCode=0
Oct 30 22:24:18.346865 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:24:18.346825    4412 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-37d062d1dd/isvc-autogluon-v1-predictor-7c64bdbd96-gjwcx" event={"ID":"8385df9b-f397-457b-b0d7-11c49429c858","Type":"ContainerDied","Data":"82dfac57dbb7a0cbdf86837804e2c3ebbfec1ad85b10e5c204cd2abc4d5045dd"}
Oct 30 22:24:18.348098 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:24:18.348081    4412 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 22:24:34.451682 ip-10-0-131-202 kubenswrapper[4412]: E1030 22:24:34.451636    4412 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bkhmz" containerName="extractor"
Oct 30 22:25:04.453608 ip-10-0-131-202 kubenswrapper[4412]: E1030 22:25:04.453569    4412 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bkhmz" containerName="extractor"
Oct 30 22:25:34.451927 ip-10-0-131-202 kubenswrapper[4412]: E1030 22:25:34.451794    4412 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bkhmz" containerName="extractor"
Oct 30 22:26:04.452889 ip-10-0-131-202 kubenswrapper[4412]: E1030 22:26:04.452841    4412 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bkhmz" containerName="extractor"
Oct 30 22:26:05.798150 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:26:05.798099    4412 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-37d062d1dd/isvc-autogluon-v1-predictor-7c64bdbd96-gjwcx" event={"ID":"8385df9b-f397-457b-b0d7-11c49429c858","Type":"ContainerStarted","Data":"2a6aff9473b6954be4fe2b113e229c71b11ea0d9252ccfc8b29afee989e986f0"}
Oct 30 22:26:05.798558 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:26:05.798171    4412 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-37d062d1dd/isvc-autogluon-v1-predictor-7c64bdbd96-gjwcx" event={"ID":"8385df9b-f397-457b-b0d7-11c49429c858","Type":"ContainerStarted","Data":"8c42e786c474fa78bc05ece15ee0db42268b8000298bd170f130f70d2d68211c"}
Oct 30 22:26:05.798558 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:26:05.798356    4412 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw0-37d062d1dd/isvc-autogluon-v1-predictor-7c64bdbd96-gjwcx"
Oct 30 22:26:06.018135 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:26:06.018064    4412 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-gw0-37d062d1dd/isvc-autogluon-v1-predictor-7c64bdbd96-gjwcx" podStartSLOduration=2.132825194 podStartE2EDuration="2m30.018028488s" totalImagesPullingTime="1m46.836196519s" totalInitContainerRuntime="41.049006775s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:23:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:26:05.967987398 +0000 UTC m=+1986.472659328" watchObservedRunningTime="2026-10-30 22:26:06.018028488 +0000 UTC m=+1986.522700437"
Oct 30 22:26:06.802536 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:26:06.802495    4412 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw0-37d062d1dd/isvc-autogluon-v1-predictor-7c64bdbd96-gjwcx"
Oct 30 22:26:06.803567 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:26:06.803543    4412 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-37d062d1dd/isvc-autogluon-v1-predictor-7c64bdbd96-gjwcx" podUID="8385df9b-f397-457b-b0d7-11c49429c858" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.23:8080: connect: connection refused"
Oct 30 22:26:07.806420 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:26:07.806377    4412 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-37d062d1dd/isvc-autogluon-v1-predictor-7c64bdbd96-gjwcx" podUID="8385df9b-f397-457b-b0d7-11c49429c858" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.23:8080: connect: connection refused"
Oct 30 22:26:12.810934 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:26:12.810903    4412 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw0-37d062d1dd/isvc-autogluon-v1-predictor-7c64bdbd96-gjwcx"
Oct 30 22:26:12.811460 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:26:12.811424    4412 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-37d062d1dd/isvc-autogluon-v1-predictor-7c64bdbd96-gjwcx" podUID="8385df9b-f397-457b-b0d7-11c49429c858" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.23:8080: connect: connection refused"
Oct 30 22:26:22.812296 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:26:22.812264    4412 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw0-37d062d1dd/isvc-autogluon-v1-predictor-7c64bdbd96-gjwcx"
Oct 30 22:26:33.730369 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:26:33.730332    4412 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw0-37d062d1dd/isvc-autogluon-v1-predictor-7c64bdbd96-gjwcx"]
Oct 30 22:26:33.730926 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:26:33.730610    4412 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw0-37d062d1dd/isvc-autogluon-v1-predictor-7c64bdbd96-gjwcx" podUID="8385df9b-f397-457b-b0d7-11c49429c858" containerName="kserve-container" containerID="cri-o://8c42e786c474fa78bc05ece15ee0db42268b8000298bd170f130f70d2d68211c" gracePeriod=30
Oct 30 22:26:33.730926 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:26:33.730711    4412 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw0-37d062d1dd/isvc-autogluon-v1-predictor-7c64bdbd96-gjwcx" podUID="8385df9b-f397-457b-b0d7-11c49429c858" containerName="kube-rbac-proxy" containerID="cri-o://2a6aff9473b6954be4fe2b113e229c71b11ea0d9252ccfc8b29afee989e986f0" gracePeriod=30
Oct 30 22:26:33.902090 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:26:33.902056    4412 generic.go:356] "Generic (PLEG): container finished" podID="8385df9b-f397-457b-b0d7-11c49429c858" containerID="2a6aff9473b6954be4fe2b113e229c71b11ea0d9252ccfc8b29afee989e986f0" exitCode=0
Oct 30 22:26:33.902247 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:26:33.902143    4412 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-37d062d1dd/isvc-autogluon-v1-predictor-7c64bdbd96-gjwcx" event={"ID":"8385df9b-f397-457b-b0d7-11c49429c858","Type":"ContainerDied","Data":"2a6aff9473b6954be4fe2b113e229c71b11ea0d9252ccfc8b29afee989e986f0"}
Oct 30 22:26:34.449724 ip-10-0-131-202 kubenswrapper[4412]: E1030 22:26:34.449680    4412 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bkhmz" containerName="extractor"
Oct 30 22:26:35.297701 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:26:35.297623    4412 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-37d062d1dd/isvc-autogluon-v1-predictor-7c64bdbd96-gjwcx"
Oct 30 22:26:35.387663 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:26:35.387632    4412 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/8385df9b-f397-457b-b0d7-11c49429c858-cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/8385df9b-f397-457b-b0d7-11c49429c858-cabundle-cert\") pod \"8385df9b-f397-457b-b0d7-11c49429c858\" (UID: \"8385df9b-f397-457b-b0d7-11c49429c858\") "
Oct 30 22:26:35.387836 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:26:35.387680    4412 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/8385df9b-f397-457b-b0d7-11c49429c858-kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/8385df9b-f397-457b-b0d7-11c49429c858-kube-rbac-proxy-sar-config\") pod \"8385df9b-f397-457b-b0d7-11c49429c858\" (UID: \"8385df9b-f397-457b-b0d7-11c49429c858\") "
Oct 30 22:26:35.387836 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:26:35.387718    4412 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8385df9b-f397-457b-b0d7-11c49429c858-kube-api-access-h8sxw\" (UniqueName: \"kubernetes.io/projected/8385df9b-f397-457b-b0d7-11c49429c858-kube-api-access-h8sxw\") pod \"8385df9b-f397-457b-b0d7-11c49429c858\" (UID: \"8385df9b-f397-457b-b0d7-11c49429c858\") "
Oct 30 22:26:35.387836 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:26:35.387741    4412 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8385df9b-f397-457b-b0d7-11c49429c858-kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/8385df9b-f397-457b-b0d7-11c49429c858-kserve-provision-location\") pod \"8385df9b-f397-457b-b0d7-11c49429c858\" (UID: \"8385df9b-f397-457b-b0d7-11c49429c858\") "
Oct 30 22:26:35.387836 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:26:35.387762    4412 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/8385df9b-f397-457b-b0d7-11c49429c858-proxy-tls\" (UniqueName: \"kubernetes.io/secret/8385df9b-f397-457b-b0d7-11c49429c858-proxy-tls\") pod \"8385df9b-f397-457b-b0d7-11c49429c858\" (UID: \"8385df9b-f397-457b-b0d7-11c49429c858\") "
Oct 30 22:26:35.388086 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:26:35.388031    4412 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8385df9b-f397-457b-b0d7-11c49429c858-cabundle-cert" pod "8385df9b-f397-457b-b0d7-11c49429c858" (UID: "8385df9b-f397-457b-b0d7-11c49429c858"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:26:35.388202 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:26:35.388177    4412 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8385df9b-f397-457b-b0d7-11c49429c858-kube-rbac-proxy-sar-config" pod "8385df9b-f397-457b-b0d7-11c49429c858" (UID: "8385df9b-f397-457b-b0d7-11c49429c858"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:26:35.390193 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:26:35.390158    4412 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8385df9b-f397-457b-b0d7-11c49429c858-proxy-tls" pod "8385df9b-f397-457b-b0d7-11c49429c858" (UID: "8385df9b-f397-457b-b0d7-11c49429c858"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:26:35.390380 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:26:35.390357    4412 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8385df9b-f397-457b-b0d7-11c49429c858-kube-api-access-h8sxw" pod "8385df9b-f397-457b-b0d7-11c49429c858" (UID: "8385df9b-f397-457b-b0d7-11c49429c858"). InnerVolumeSpecName "kube-api-access-h8sxw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:26:35.392399 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:26:35.392381    4412 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8385df9b-f397-457b-b0d7-11c49429c858-kserve-provision-location" pod "8385df9b-f397-457b-b0d7-11c49429c858" (UID: "8385df9b-f397-457b-b0d7-11c49429c858"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:26:35.488369 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:26:35.488328    4412 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h8sxw\" (UniqueName: \"kubernetes.io/projected/8385df9b-f397-457b-b0d7-11c49429c858-kube-api-access-h8sxw\") on node \"ip-10-0-131-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:26:35.488369 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:26:35.488363    4412 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/8385df9b-f397-457b-b0d7-11c49429c858-kserve-provision-location\") on node \"ip-10-0-131-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:26:35.488369 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:26:35.488377    4412 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/8385df9b-f397-457b-b0d7-11c49429c858-proxy-tls\") on node \"ip-10-0-131-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:26:35.488613 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:26:35.488389    4412 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/8385df9b-f397-457b-b0d7-11c49429c858-cabundle-cert\") on node \"ip-10-0-131-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:26:35.488613 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:26:35.488404    4412 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/8385df9b-f397-457b-b0d7-11c49429c858-kube-rbac-proxy-sar-config\") on node \"ip-10-0-131-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:26:35.912403 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:26:35.912323    4412 generic.go:356] "Generic (PLEG): container finished" podID="8385df9b-f397-457b-b0d7-11c49429c858" containerID="8c42e786c474fa78bc05ece15ee0db42268b8000298bd170f130f70d2d68211c" exitCode=0
Oct 30 22:26:35.912548 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:26:35.912402    4412 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-37d062d1dd/isvc-autogluon-v1-predictor-7c64bdbd96-gjwcx"
Oct 30 22:26:35.912548 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:26:35.912410    4412 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-37d062d1dd/isvc-autogluon-v1-predictor-7c64bdbd96-gjwcx" event={"ID":"8385df9b-f397-457b-b0d7-11c49429c858","Type":"ContainerDied","Data":"8c42e786c474fa78bc05ece15ee0db42268b8000298bd170f130f70d2d68211c"}
Oct 30 22:26:35.912548 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:26:35.912493    4412 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-37d062d1dd/isvc-autogluon-v1-predictor-7c64bdbd96-gjwcx" event={"ID":"8385df9b-f397-457b-b0d7-11c49429c858","Type":"ContainerDied","Data":"2b42d8f5360bf51d9764a98efea28263ec2af511b3835010a8049ff08a5f70c2"}
Oct 30 22:26:35.912548 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:26:35.912520    4412 scope.go:122] "RemoveContainer" containerID="2a6aff9473b6954be4fe2b113e229c71b11ea0d9252ccfc8b29afee989e986f0"
Oct 30 22:26:35.920542 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:26:35.920525    4412 scope.go:122] "RemoveContainer" containerID="8c42e786c474fa78bc05ece15ee0db42268b8000298bd170f130f70d2d68211c"
Oct 30 22:26:35.927915 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:26:35.927893    4412 scope.go:122] "RemoveContainer" containerID="82dfac57dbb7a0cbdf86837804e2c3ebbfec1ad85b10e5c204cd2abc4d5045dd"
Oct 30 22:26:35.935267 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:26:35.935248    4412 scope.go:122] "RemoveContainer" containerID="2a6aff9473b6954be4fe2b113e229c71b11ea0d9252ccfc8b29afee989e986f0"
Oct 30 22:26:35.935570 ip-10-0-131-202 kubenswrapper[4412]: E1030 22:26:35.935550    4412 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2a6aff9473b6954be4fe2b113e229c71b11ea0d9252ccfc8b29afee989e986f0\": container with ID starting with 2a6aff9473b6954be4fe2b113e229c71b11ea0d9252ccfc8b29afee989e986f0 not found: ID does not exist" containerID="2a6aff9473b6954be4fe2b113e229c71b11ea0d9252ccfc8b29afee989e986f0"
Oct 30 22:26:35.935615 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:26:35.935581    4412 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2a6aff9473b6954be4fe2b113e229c71b11ea0d9252ccfc8b29afee989e986f0"} err="failed to get container status \"2a6aff9473b6954be4fe2b113e229c71b11ea0d9252ccfc8b29afee989e986f0\": rpc error: code = NotFound desc = could not find container \"2a6aff9473b6954be4fe2b113e229c71b11ea0d9252ccfc8b29afee989e986f0\": container with ID starting with 2a6aff9473b6954be4fe2b113e229c71b11ea0d9252ccfc8b29afee989e986f0 not found: ID does not exist"
Oct 30 22:26:35.935615 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:26:35.935598    4412 scope.go:122] "RemoveContainer" containerID="8c42e786c474fa78bc05ece15ee0db42268b8000298bd170f130f70d2d68211c"
Oct 30 22:26:35.935862 ip-10-0-131-202 kubenswrapper[4412]: E1030 22:26:35.935845    4412 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8c42e786c474fa78bc05ece15ee0db42268b8000298bd170f130f70d2d68211c\": container with ID starting with 8c42e786c474fa78bc05ece15ee0db42268b8000298bd170f130f70d2d68211c not found: ID does not exist" containerID="8c42e786c474fa78bc05ece15ee0db42268b8000298bd170f130f70d2d68211c"
Oct 30 22:26:35.935905 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:26:35.935869    4412 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8c42e786c474fa78bc05ece15ee0db42268b8000298bd170f130f70d2d68211c"} err="failed to get container status \"8c42e786c474fa78bc05ece15ee0db42268b8000298bd170f130f70d2d68211c\": rpc error: code = NotFound desc = could not find container \"8c42e786c474fa78bc05ece15ee0db42268b8000298bd170f130f70d2d68211c\": container with ID starting with 8c42e786c474fa78bc05ece15ee0db42268b8000298bd170f130f70d2d68211c not found: ID does not exist"
Oct 30 22:26:35.935905 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:26:35.935884    4412 scope.go:122] "RemoveContainer" containerID="82dfac57dbb7a0cbdf86837804e2c3ebbfec1ad85b10e5c204cd2abc4d5045dd"
Oct 30 22:26:35.936128 ip-10-0-131-202 kubenswrapper[4412]: E1030 22:26:35.936100    4412 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"82dfac57dbb7a0cbdf86837804e2c3ebbfec1ad85b10e5c204cd2abc4d5045dd\": container with ID starting with 82dfac57dbb7a0cbdf86837804e2c3ebbfec1ad85b10e5c204cd2abc4d5045dd not found: ID does not exist" containerID="82dfac57dbb7a0cbdf86837804e2c3ebbfec1ad85b10e5c204cd2abc4d5045dd"
Oct 30 22:26:35.936170 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:26:35.936142    4412 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"82dfac57dbb7a0cbdf86837804e2c3ebbfec1ad85b10e5c204cd2abc4d5045dd"} err="failed to get container status \"82dfac57dbb7a0cbdf86837804e2c3ebbfec1ad85b10e5c204cd2abc4d5045dd\": rpc error: code = NotFound desc = could not find container \"82dfac57dbb7a0cbdf86837804e2c3ebbfec1ad85b10e5c204cd2abc4d5045dd\": container with ID starting with 82dfac57dbb7a0cbdf86837804e2c3ebbfec1ad85b10e5c204cd2abc4d5045dd not found: ID does not exist"
Oct 30 22:26:36.131388 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:26:36.131355    4412 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw0-37d062d1dd/isvc-autogluon-v1-predictor-7c64bdbd96-gjwcx"]
Oct 30 22:26:36.181973 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:26:36.181901    4412 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-gw0-37d062d1dd/isvc-autogluon-v1-predictor-7c64bdbd96-gjwcx"]
Oct 30 22:26:37.626259 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:26:37.626221    4412 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="8385df9b-f397-457b-b0d7-11c49429c858" path="/var/lib/kubelet/pods/8385df9b-f397-457b-b0d7-11c49429c858/volumes"
Oct 30 22:27:04.448143 ip-10-0-131-202 kubenswrapper[4412]: E1030 22:27:04.448087    4412 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bkhmz" containerName="extractor"
Oct 30 22:27:34.447990 ip-10-0-131-202 kubenswrapper[4412]: E1030 22:27:34.447942    4412 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bkhmz" containerName="extractor"
Oct 30 22:28:04.447901 ip-10-0-131-202 kubenswrapper[4412]: E1030 22:28:04.447859    4412 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bkhmz" containerName="extractor"
Oct 30 22:28:34.449596 ip-10-0-131-202 kubenswrapper[4412]: E1030 22:28:34.449553    4412 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bkhmz" containerName="extractor"
Oct 30 22:28:41.771073 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:28:41.771039    4412 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-gw1-7476d02544/isvc-tensorflow-predictor-6c447b9699-llp7t"]
Oct 30 22:28:41.771558 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:28:41.771341    4412 cpu_manager.go:414] "containerMap: removing container" podUID="8385df9b-f397-457b-b0d7-11c49429c858" containerName="kserve-container"
Oct 30 22:28:41.771558 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:28:41.771353    4412 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8385df9b-f397-457b-b0d7-11c49429c858" containerName="kserve-container"
Oct 30 22:28:41.771558 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:28:41.771366    4412 cpu_manager.go:414] "containerMap: removing container" podUID="8385df9b-f397-457b-b0d7-11c49429c858" containerName="storage-initializer"
Oct 30 22:28:41.771558 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:28:41.771372    4412 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8385df9b-f397-457b-b0d7-11c49429c858" containerName="storage-initializer"
Oct 30 22:28:41.771558 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:28:41.771391    4412 cpu_manager.go:414] "containerMap: removing container" podUID="8385df9b-f397-457b-b0d7-11c49429c858" containerName="kube-rbac-proxy"
Oct 30 22:28:41.771558 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:28:41.771396    4412 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8385df9b-f397-457b-b0d7-11c49429c858" containerName="kube-rbac-proxy"
Oct 30 22:28:41.771558 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:28:41.771438    4412 memory_manager.go:365] "RemoveStaleState removing state" podUID="8385df9b-f397-457b-b0d7-11c49429c858" containerName="kserve-container"
Oct 30 22:28:41.771558 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:28:41.771450    4412 memory_manager.go:365] "RemoveStaleState removing state" podUID="8385df9b-f397-457b-b0d7-11c49429c858" containerName="kube-rbac-proxy"
Oct 30 22:28:41.773593 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:28:41.773569    4412 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-7476d02544/isvc-tensorflow-predictor-6c447b9699-llp7t"
Oct 30 22:28:41.826703 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:28:41.826673    4412 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw1-7476d02544\"/\"odh-kserve-custom-ca-bundle\""
Oct 30 22:28:41.837167 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:28:41.837142    4412 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/3b69d7af-4e03-477d-bca1-36e7dcaca0df-proxy-tls\") pod \"isvc-tensorflow-predictor-6c447b9699-llp7t\" (UID: \"3b69d7af-4e03-477d-bca1-36e7dcaca0df\") " pod="e2e-gw1-7476d02544/isvc-tensorflow-predictor-6c447b9699-llp7t"
Oct 30 22:28:41.837256 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:28:41.837202    4412 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/3b69d7af-4e03-477d-bca1-36e7dcaca0df-cabundle-cert\") pod \"isvc-tensorflow-predictor-6c447b9699-llp7t\" (UID: \"3b69d7af-4e03-477d-bca1-36e7dcaca0df\") " pod="e2e-gw1-7476d02544/isvc-tensorflow-predictor-6c447b9699-llp7t"
Oct 30 22:28:41.837295 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:28:41.837253    4412 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/3b69d7af-4e03-477d-bca1-36e7dcaca0df-kube-rbac-proxy-sar-config\") pod \"isvc-tensorflow-predictor-6c447b9699-llp7t\" (UID: \"3b69d7af-4e03-477d-bca1-36e7dcaca0df\") " pod="e2e-gw1-7476d02544/isvc-tensorflow-predictor-6c447b9699-llp7t"
Oct 30 22:28:41.837329 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:28:41.837305    4412 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v57nc\" (UniqueName: \"kubernetes.io/projected/3b69d7af-4e03-477d-bca1-36e7dcaca0df-kube-api-access-v57nc\") pod \"isvc-tensorflow-predictor-6c447b9699-llp7t\" (UID: \"3b69d7af-4e03-477d-bca1-36e7dcaca0df\") " pod="e2e-gw1-7476d02544/isvc-tensorflow-predictor-6c447b9699-llp7t"
Oct 30 22:28:41.837362 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:28:41.837325    4412 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/3b69d7af-4e03-477d-bca1-36e7dcaca0df-kserve-provision-location\") pod \"isvc-tensorflow-predictor-6c447b9699-llp7t\" (UID: \"3b69d7af-4e03-477d-bca1-36e7dcaca0df\") " pod="e2e-gw1-7476d02544/isvc-tensorflow-predictor-6c447b9699-llp7t"
Oct 30 22:28:41.877081 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:28:41.877040    4412 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw1-7476d02544\"/\"seaweedfs-s3-creds\""
Oct 30 22:28:41.877081 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:28:41.877050    4412 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw1-7476d02544\"/\"kube-root-ca.crt\""
Oct 30 22:28:41.877301 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:28:41.877108    4412 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw1-7476d02544\"/\"openshift-service-ca.crt\""
Oct 30 22:28:41.877301 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:28:41.877158    4412 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw1-7476d02544\"/\"isvc-tensorflow-kube-rbac-proxy-sar-config\""
Oct 30 22:28:41.877301 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:28:41.877174    4412 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw1-7476d02544\"/\"default-dockercfg-cxfkn\""
Oct 30 22:28:41.877453 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:28:41.877329    4412 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw1-7476d02544\"/\"isvc-tensorflow-predictor-serving-cert\""
Oct 30 22:28:41.878673 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:28:41.878650    4412 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw1-7476d02544/isvc-tensorflow-predictor-6c447b9699-llp7t"]
Oct 30 22:28:41.938047 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:28:41.938014    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/3b69d7af-4e03-477d-bca1-36e7dcaca0df-proxy-tls\") pod \"isvc-tensorflow-predictor-6c447b9699-llp7t\" (UID: \"3b69d7af-4e03-477d-bca1-36e7dcaca0df\") " pod="e2e-gw1-7476d02544/isvc-tensorflow-predictor-6c447b9699-llp7t"
Oct 30 22:28:41.938047 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:28:41.938051    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/3b69d7af-4e03-477d-bca1-36e7dcaca0df-cabundle-cert\") pod \"isvc-tensorflow-predictor-6c447b9699-llp7t\" (UID: \"3b69d7af-4e03-477d-bca1-36e7dcaca0df\") " pod="e2e-gw1-7476d02544/isvc-tensorflow-predictor-6c447b9699-llp7t"
Oct 30 22:28:41.938304 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:28:41.938106    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/3b69d7af-4e03-477d-bca1-36e7dcaca0df-kube-rbac-proxy-sar-config\") pod \"isvc-tensorflow-predictor-6c447b9699-llp7t\" (UID: \"3b69d7af-4e03-477d-bca1-36e7dcaca0df\") " pod="e2e-gw1-7476d02544/isvc-tensorflow-predictor-6c447b9699-llp7t"
Oct 30 22:28:41.938304 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:28:41.938168    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v57nc\" (UniqueName: \"kubernetes.io/projected/3b69d7af-4e03-477d-bca1-36e7dcaca0df-kube-api-access-v57nc\") pod \"isvc-tensorflow-predictor-6c447b9699-llp7t\" (UID: \"3b69d7af-4e03-477d-bca1-36e7dcaca0df\") " pod="e2e-gw1-7476d02544/isvc-tensorflow-predictor-6c447b9699-llp7t"
Oct 30 22:28:41.938304 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:28:41.938193    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/3b69d7af-4e03-477d-bca1-36e7dcaca0df-kserve-provision-location\") pod \"isvc-tensorflow-predictor-6c447b9699-llp7t\" (UID: \"3b69d7af-4e03-477d-bca1-36e7dcaca0df\") " pod="e2e-gw1-7476d02544/isvc-tensorflow-predictor-6c447b9699-llp7t"
Oct 30 22:28:41.938666 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:28:41.938631    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/3b69d7af-4e03-477d-bca1-36e7dcaca0df-kserve-provision-location\") pod \"isvc-tensorflow-predictor-6c447b9699-llp7t\" (UID: \"3b69d7af-4e03-477d-bca1-36e7dcaca0df\") " pod="e2e-gw1-7476d02544/isvc-tensorflow-predictor-6c447b9699-llp7t"
Oct 30 22:28:41.938866 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:28:41.938845    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/3b69d7af-4e03-477d-bca1-36e7dcaca0df-cabundle-cert\") pod \"isvc-tensorflow-predictor-6c447b9699-llp7t\" (UID: \"3b69d7af-4e03-477d-bca1-36e7dcaca0df\") " pod="e2e-gw1-7476d02544/isvc-tensorflow-predictor-6c447b9699-llp7t"
Oct 30 22:28:41.938933 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:28:41.938847    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/3b69d7af-4e03-477d-bca1-36e7dcaca0df-kube-rbac-proxy-sar-config\") pod \"isvc-tensorflow-predictor-6c447b9699-llp7t\" (UID: \"3b69d7af-4e03-477d-bca1-36e7dcaca0df\") " pod="e2e-gw1-7476d02544/isvc-tensorflow-predictor-6c447b9699-llp7t"
Oct 30 22:28:41.940727 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:28:41.940708    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/3b69d7af-4e03-477d-bca1-36e7dcaca0df-proxy-tls\") pod \"isvc-tensorflow-predictor-6c447b9699-llp7t\" (UID: \"3b69d7af-4e03-477d-bca1-36e7dcaca0df\") " pod="e2e-gw1-7476d02544/isvc-tensorflow-predictor-6c447b9699-llp7t"
Oct 30 22:28:41.997514 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:28:41.997483    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v57nc\" (UniqueName: \"kubernetes.io/projected/3b69d7af-4e03-477d-bca1-36e7dcaca0df-kube-api-access-v57nc\") pod \"isvc-tensorflow-predictor-6c447b9699-llp7t\" (UID: \"3b69d7af-4e03-477d-bca1-36e7dcaca0df\") " pod="e2e-gw1-7476d02544/isvc-tensorflow-predictor-6c447b9699-llp7t"
Oct 30 22:28:42.087082 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:28:42.086994    4412 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-7476d02544/isvc-tensorflow-predictor-6c447b9699-llp7t"
Oct 30 22:28:42.509764 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:28:42.509739    4412 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw1-7476d02544/isvc-tensorflow-predictor-6c447b9699-llp7t"]
Oct 30 22:28:42.511648 ip-10-0-131-202 kubenswrapper[4412]: W1030 22:28:42.511619    4412 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3b69d7af_4e03_477d_bca1_36e7dcaca0df.slice/crio-8da30888b492926f0b0505cfb30c5d11e5e0412ac4e0100cde73be2d9a9afa4d WatchSource:0}: Error finding container 8da30888b492926f0b0505cfb30c5d11e5e0412ac4e0100cde73be2d9a9afa4d: Status 404 returned error can't find the container with id 8da30888b492926f0b0505cfb30c5d11e5e0412ac4e0100cde73be2d9a9afa4d
Oct 30 22:28:43.363626 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:28:43.363590    4412 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-7476d02544/isvc-tensorflow-predictor-6c447b9699-llp7t" event={"ID":"3b69d7af-4e03-477d-bca1-36e7dcaca0df","Type":"ContainerStarted","Data":"7b1da1f8d4ce76c551db6a70ef9f561057fe87cbd6c8d0d40f0c5af4a7c6fa6d"}
Oct 30 22:28:43.364096 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:28:43.363650    4412 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-7476d02544/isvc-tensorflow-predictor-6c447b9699-llp7t" event={"ID":"3b69d7af-4e03-477d-bca1-36e7dcaca0df","Type":"ContainerStarted","Data":"8da30888b492926f0b0505cfb30c5d11e5e0412ac4e0100cde73be2d9a9afa4d"}
Oct 30 22:28:48.383158 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:28:48.383111    4412 generic.go:356] "Generic (PLEG): container finished" podID="3b69d7af-4e03-477d-bca1-36e7dcaca0df" containerID="7b1da1f8d4ce76c551db6a70ef9f561057fe87cbd6c8d0d40f0c5af4a7c6fa6d" exitCode=0
Oct 30 22:28:48.383608 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:28:48.383185    4412 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-7476d02544/isvc-tensorflow-predictor-6c447b9699-llp7t" event={"ID":"3b69d7af-4e03-477d-bca1-36e7dcaca0df","Type":"ContainerDied","Data":"7b1da1f8d4ce76c551db6a70ef9f561057fe87cbd6c8d0d40f0c5af4a7c6fa6d"}
Oct 30 22:28:53.406949 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:28:53.406914    4412 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-7476d02544/isvc-tensorflow-predictor-6c447b9699-llp7t" event={"ID":"3b69d7af-4e03-477d-bca1-36e7dcaca0df","Type":"ContainerStarted","Data":"c53584594e0808ead253658f79f87eb2622dedd48d8e93c147a377f6b24e22fc"}
Oct 30 22:28:53.407409 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:28:53.406979    4412 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-7476d02544/isvc-tensorflow-predictor-6c447b9699-llp7t" event={"ID":"3b69d7af-4e03-477d-bca1-36e7dcaca0df","Type":"ContainerStarted","Data":"8ad5d683bf690573b4c4dd5e6b4fb717421c330b732290edc8cb150d437f929d"}
Oct 30 22:28:53.407409 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:28:53.407161    4412 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw1-7476d02544/isvc-tensorflow-predictor-6c447b9699-llp7t"
Oct 30 22:28:54.411273 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:28:54.411243    4412 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw1-7476d02544/isvc-tensorflow-predictor-6c447b9699-llp7t"
Oct 30 22:28:54.412699 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:28:54.412669    4412 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-7476d02544/isvc-tensorflow-predictor-6c447b9699-llp7t" podUID="3b69d7af-4e03-477d-bca1-36e7dcaca0df" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.24:8080: connect: connection refused"
Oct 30 22:28:55.415904 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:28:55.415869    4412 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-7476d02544/isvc-tensorflow-predictor-6c447b9699-llp7t" podUID="3b69d7af-4e03-477d-bca1-36e7dcaca0df" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.24:8080: connect: connection refused"
Oct 30 22:29:00.420626 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:29:00.420597    4412 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw1-7476d02544/isvc-tensorflow-predictor-6c447b9699-llp7t"
Oct 30 22:29:00.421297 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:29:00.421269    4412 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-7476d02544/isvc-tensorflow-predictor-6c447b9699-llp7t" podUID="3b69d7af-4e03-477d-bca1-36e7dcaca0df" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.24:8080: connect: connection refused"
Oct 30 22:29:00.592177 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:29:00.592093    4412 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-gw1-7476d02544/isvc-tensorflow-predictor-6c447b9699-llp7t" podStartSLOduration=9.464598482 podStartE2EDuration="19.592081195s" totalImagesPullingTime="4.807434555s" totalInitContainerRuntime="5.320048158s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:28:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:28:53.583750494 +0000 UTC m=+2154.088422447" watchObservedRunningTime="2026-10-30 22:29:00.592081195 +0000 UTC m=+2161.096753150"
Oct 30 22:29:04.449478 ip-10-0-131-202 kubenswrapper[4412]: E1030 22:29:04.449441    4412 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bkhmz" containerName="extractor"
Oct 30 22:29:10.422267 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:29:10.422233    4412 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw1-7476d02544/isvc-tensorflow-predictor-6c447b9699-llp7t"
Oct 30 22:29:32.397052 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:29:32.396977    4412 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw1-7476d02544/isvc-tensorflow-predictor-6c447b9699-llp7t"]
Oct 30 22:29:32.397465 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:29:32.397233    4412 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw1-7476d02544/isvc-tensorflow-predictor-6c447b9699-llp7t" podUID="3b69d7af-4e03-477d-bca1-36e7dcaca0df" containerName="kserve-container" containerID="cri-o://8ad5d683bf690573b4c4dd5e6b4fb717421c330b732290edc8cb150d437f929d" gracePeriod=30
Oct 30 22:29:32.397465 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:29:32.397284    4412 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw1-7476d02544/isvc-tensorflow-predictor-6c447b9699-llp7t" podUID="3b69d7af-4e03-477d-bca1-36e7dcaca0df" containerName="kube-rbac-proxy" containerID="cri-o://c53584594e0808ead253658f79f87eb2622dedd48d8e93c147a377f6b24e22fc" gracePeriod=30
Oct 30 22:29:32.554325 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:29:32.554292    4412 generic.go:356] "Generic (PLEG): container finished" podID="3b69d7af-4e03-477d-bca1-36e7dcaca0df" containerID="c53584594e0808ead253658f79f87eb2622dedd48d8e93c147a377f6b24e22fc" exitCode=0
Oct 30 22:29:32.554471 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:29:32.554364    4412 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-7476d02544/isvc-tensorflow-predictor-6c447b9699-llp7t" event={"ID":"3b69d7af-4e03-477d-bca1-36e7dcaca0df","Type":"ContainerDied","Data":"c53584594e0808ead253658f79f87eb2622dedd48d8e93c147a377f6b24e22fc"}
Oct 30 22:29:34.448594 ip-10-0-131-202 kubenswrapper[4412]: E1030 22:29:34.448560    4412 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bkhmz" containerName="extractor"
Oct 30 22:29:35.416284 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:29:35.416239    4412 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-7476d02544/isvc-tensorflow-predictor-6c447b9699-llp7t" podUID="3b69d7af-4e03-477d-bca1-36e7dcaca0df" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.24:8643/healthz\": dial tcp 10.133.0.24:8643: connect: connection refused"
Oct 30 22:29:40.416156 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:29:40.416101    4412 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-7476d02544/isvc-tensorflow-predictor-6c447b9699-llp7t" podUID="3b69d7af-4e03-477d-bca1-36e7dcaca0df" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.24:8643/healthz\": dial tcp 10.133.0.24:8643: connect: connection refused"
Oct 30 22:29:45.416561 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:29:45.416471    4412 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-7476d02544/isvc-tensorflow-predictor-6c447b9699-llp7t" podUID="3b69d7af-4e03-477d-bca1-36e7dcaca0df" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.24:8643/healthz\": dial tcp 10.133.0.24:8643: connect: connection refused"
Oct 30 22:29:45.416958 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:29:45.416641    4412 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw1-7476d02544/isvc-tensorflow-predictor-6c447b9699-llp7t"
Oct 30 22:29:50.416170 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:29:50.416106    4412 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-7476d02544/isvc-tensorflow-predictor-6c447b9699-llp7t" podUID="3b69d7af-4e03-477d-bca1-36e7dcaca0df" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.24:8643/healthz\": dial tcp 10.133.0.24:8643: connect: connection refused"
Oct 30 22:29:55.416687 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:29:55.416645    4412 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-7476d02544/isvc-tensorflow-predictor-6c447b9699-llp7t" podUID="3b69d7af-4e03-477d-bca1-36e7dcaca0df" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.24:8643/healthz\": dial tcp 10.133.0.24:8643: connect: connection refused"
Oct 30 22:30:00.416024 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:30:00.415973    4412 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-7476d02544/isvc-tensorflow-predictor-6c447b9699-llp7t" podUID="3b69d7af-4e03-477d-bca1-36e7dcaca0df" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.24:8643/healthz\": dial tcp 10.133.0.24:8643: connect: connection refused"
Oct 30 22:30:02.664966 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:30:02.664861    4412 generic.go:356] "Generic (PLEG): container finished" podID="3b69d7af-4e03-477d-bca1-36e7dcaca0df" containerID="8ad5d683bf690573b4c4dd5e6b4fb717421c330b732290edc8cb150d437f929d" exitCode=137
Oct 30 22:30:02.664966 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:30:02.664904    4412 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-7476d02544/isvc-tensorflow-predictor-6c447b9699-llp7t" event={"ID":"3b69d7af-4e03-477d-bca1-36e7dcaca0df","Type":"ContainerDied","Data":"8ad5d683bf690573b4c4dd5e6b4fb717421c330b732290edc8cb150d437f929d"}
Oct 30 22:30:03.034496 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:30:03.034469    4412 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-7476d02544/isvc-tensorflow-predictor-6c447b9699-llp7t"
Oct 30 22:30:03.197060 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:30:03.197015    4412 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/3b69d7af-4e03-477d-bca1-36e7dcaca0df-proxy-tls\" (UniqueName: \"kubernetes.io/secret/3b69d7af-4e03-477d-bca1-36e7dcaca0df-proxy-tls\") pod \"3b69d7af-4e03-477d-bca1-36e7dcaca0df\" (UID: \"3b69d7af-4e03-477d-bca1-36e7dcaca0df\") "
Oct 30 22:30:03.197382 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:30:03.197148    4412 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3b69d7af-4e03-477d-bca1-36e7dcaca0df-kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/3b69d7af-4e03-477d-bca1-36e7dcaca0df-kserve-provision-location\") pod \"3b69d7af-4e03-477d-bca1-36e7dcaca0df\" (UID: \"3b69d7af-4e03-477d-bca1-36e7dcaca0df\") "
Oct 30 22:30:03.197382 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:30:03.197185    4412 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/3b69d7af-4e03-477d-bca1-36e7dcaca0df-cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/3b69d7af-4e03-477d-bca1-36e7dcaca0df-cabundle-cert\") pod \"3b69d7af-4e03-477d-bca1-36e7dcaca0df\" (UID: \"3b69d7af-4e03-477d-bca1-36e7dcaca0df\") "
Oct 30 22:30:03.197382 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:30:03.197209    4412 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3b69d7af-4e03-477d-bca1-36e7dcaca0df-kube-api-access-v57nc\" (UniqueName: \"kubernetes.io/projected/3b69d7af-4e03-477d-bca1-36e7dcaca0df-kube-api-access-v57nc\") pod \"3b69d7af-4e03-477d-bca1-36e7dcaca0df\" (UID: \"3b69d7af-4e03-477d-bca1-36e7dcaca0df\") "
Oct 30 22:30:03.197382 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:30:03.197245    4412 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/3b69d7af-4e03-477d-bca1-36e7dcaca0df-kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/3b69d7af-4e03-477d-bca1-36e7dcaca0df-kube-rbac-proxy-sar-config\") pod \"3b69d7af-4e03-477d-bca1-36e7dcaca0df\" (UID: \"3b69d7af-4e03-477d-bca1-36e7dcaca0df\") "
Oct 30 22:30:03.197805 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:30:03.197593    4412 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3b69d7af-4e03-477d-bca1-36e7dcaca0df-cabundle-cert" pod "3b69d7af-4e03-477d-bca1-36e7dcaca0df" (UID: "3b69d7af-4e03-477d-bca1-36e7dcaca0df"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:30:03.197805 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:30:03.197626    4412 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3b69d7af-4e03-477d-bca1-36e7dcaca0df-kube-rbac-proxy-sar-config" pod "3b69d7af-4e03-477d-bca1-36e7dcaca0df" (UID: "3b69d7af-4e03-477d-bca1-36e7dcaca0df"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:30:03.199576 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:30:03.199556    4412 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3b69d7af-4e03-477d-bca1-36e7dcaca0df-kube-api-access-v57nc" pod "3b69d7af-4e03-477d-bca1-36e7dcaca0df" (UID: "3b69d7af-4e03-477d-bca1-36e7dcaca0df"). InnerVolumeSpecName "kube-api-access-v57nc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:30:03.199644 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:30:03.199631    4412 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3b69d7af-4e03-477d-bca1-36e7dcaca0df-proxy-tls" pod "3b69d7af-4e03-477d-bca1-36e7dcaca0df" (UID: "3b69d7af-4e03-477d-bca1-36e7dcaca0df"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:30:03.208512 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:30:03.208487    4412 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3b69d7af-4e03-477d-bca1-36e7dcaca0df-kserve-provision-location" pod "3b69d7af-4e03-477d-bca1-36e7dcaca0df" (UID: "3b69d7af-4e03-477d-bca1-36e7dcaca0df"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:30:03.298531 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:30:03.298458    4412 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/3b69d7af-4e03-477d-bca1-36e7dcaca0df-proxy-tls\") on node \"ip-10-0-131-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:30:03.298531 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:30:03.298485    4412 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/3b69d7af-4e03-477d-bca1-36e7dcaca0df-kserve-provision-location\") on node \"ip-10-0-131-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:30:03.298531 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:30:03.298499    4412 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/3b69d7af-4e03-477d-bca1-36e7dcaca0df-cabundle-cert\") on node \"ip-10-0-131-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:30:03.298531 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:30:03.298511    4412 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v57nc\" (UniqueName: \"kubernetes.io/projected/3b69d7af-4e03-477d-bca1-36e7dcaca0df-kube-api-access-v57nc\") on node \"ip-10-0-131-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:30:03.298531 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:30:03.298524    4412 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/3b69d7af-4e03-477d-bca1-36e7dcaca0df-kube-rbac-proxy-sar-config\") on node \"ip-10-0-131-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:30:03.670728 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:30:03.670695    4412 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-7476d02544/isvc-tensorflow-predictor-6c447b9699-llp7t" event={"ID":"3b69d7af-4e03-477d-bca1-36e7dcaca0df","Type":"ContainerDied","Data":"8da30888b492926f0b0505cfb30c5d11e5e0412ac4e0100cde73be2d9a9afa4d"}
Oct 30 22:30:03.671162 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:30:03.670734    4412 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-7476d02544/isvc-tensorflow-predictor-6c447b9699-llp7t"
Oct 30 22:30:03.671162 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:30:03.670757    4412 scope.go:122] "RemoveContainer" containerID="c53584594e0808ead253658f79f87eb2622dedd48d8e93c147a377f6b24e22fc"
Oct 30 22:30:03.680899 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:30:03.680877    4412 scope.go:122] "RemoveContainer" containerID="8ad5d683bf690573b4c4dd5e6b4fb717421c330b732290edc8cb150d437f929d"
Oct 30 22:30:03.688940 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:30:03.688918    4412 scope.go:122] "RemoveContainer" containerID="7b1da1f8d4ce76c551db6a70ef9f561057fe87cbd6c8d0d40f0c5af4a7c6fa6d"
Oct 30 22:30:03.838482 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:30:03.838454    4412 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw1-7476d02544/isvc-tensorflow-predictor-6c447b9699-llp7t"]
Oct 30 22:30:03.889218 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:30:03.889148    4412 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-gw1-7476d02544/isvc-tensorflow-predictor-6c447b9699-llp7t"]
Oct 30 22:30:04.448039 ip-10-0-131-202 kubenswrapper[4412]: E1030 22:30:04.447995    4412 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bkhmz" containerName="extractor"
Oct 30 22:30:05.625221 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:30:05.625179    4412 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="3b69d7af-4e03-477d-bca1-36e7dcaca0df" path="/var/lib/kubelet/pods/3b69d7af-4e03-477d-bca1-36e7dcaca0df/volumes"
Oct 30 22:30:07.940840 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:30:07.940807    4412 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-gw1-7476d02544/isvc-tensorflow-runtime-predictor-bf6c5ccbc-qvlch"]
Oct 30 22:30:07.941337 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:30:07.941085    4412 cpu_manager.go:414] "containerMap: removing container" podUID="3b69d7af-4e03-477d-bca1-36e7dcaca0df" containerName="kserve-container"
Oct 30 22:30:07.941337 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:30:07.941095    4412 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3b69d7af-4e03-477d-bca1-36e7dcaca0df" containerName="kserve-container"
Oct 30 22:30:07.941337 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:30:07.941104    4412 cpu_manager.go:414] "containerMap: removing container" podUID="3b69d7af-4e03-477d-bca1-36e7dcaca0df" containerName="kube-rbac-proxy"
Oct 30 22:30:07.941337 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:30:07.941109    4412 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3b69d7af-4e03-477d-bca1-36e7dcaca0df" containerName="kube-rbac-proxy"
Oct 30 22:30:07.941337 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:30:07.941144    4412 cpu_manager.go:414] "containerMap: removing container" podUID="3b69d7af-4e03-477d-bca1-36e7dcaca0df" containerName="storage-initializer"
Oct 30 22:30:07.941337 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:30:07.941149    4412 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3b69d7af-4e03-477d-bca1-36e7dcaca0df" containerName="storage-initializer"
Oct 30 22:30:07.941337 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:30:07.941192    4412 memory_manager.go:365] "RemoveStaleState removing state" podUID="3b69d7af-4e03-477d-bca1-36e7dcaca0df" containerName="kserve-container"
Oct 30 22:30:07.941337 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:30:07.941198    4412 memory_manager.go:365] "RemoveStaleState removing state" podUID="3b69d7af-4e03-477d-bca1-36e7dcaca0df" containerName="kube-rbac-proxy"
Oct 30 22:30:07.944318 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:30:07.944303    4412 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-7476d02544/isvc-tensorflow-runtime-predictor-bf6c5ccbc-qvlch"
Oct 30 22:30:07.998284 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:30:07.998252    4412 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw1-7476d02544\"/\"isvc-tensorflow-runtime-kube-rbac-proxy-sar-config\""
Oct 30 22:30:07.998427 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:30:07.998324    4412 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw1-7476d02544\"/\"odh-kserve-custom-ca-bundle\""
Oct 30 22:30:07.998427 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:30:07.998329    4412 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw1-7476d02544\"/\"seaweedfs-s3-creds\""
Oct 30 22:30:08.037110 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:30:08.037083    4412 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/b0f81836-305f-4f94-87ec-d24c98cae445-proxy-tls\") pod \"isvc-tensorflow-runtime-predictor-bf6c5ccbc-qvlch\" (UID: \"b0f81836-305f-4f94-87ec-d24c98cae445\") " pod="e2e-gw1-7476d02544/isvc-tensorflow-runtime-predictor-bf6c5ccbc-qvlch"
Oct 30 22:30:08.037264 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:30:08.037138    4412 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7g95q\" (UniqueName: \"kubernetes.io/projected/b0f81836-305f-4f94-87ec-d24c98cae445-kube-api-access-7g95q\") pod \"isvc-tensorflow-runtime-predictor-bf6c5ccbc-qvlch\" (UID: \"b0f81836-305f-4f94-87ec-d24c98cae445\") " pod="e2e-gw1-7476d02544/isvc-tensorflow-runtime-predictor-bf6c5ccbc-qvlch"
Oct 30 22:30:08.037264 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:30:08.037228    4412 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/b0f81836-305f-4f94-87ec-d24c98cae445-kube-rbac-proxy-sar-config\") pod \"isvc-tensorflow-runtime-predictor-bf6c5ccbc-qvlch\" (UID: \"b0f81836-305f-4f94-87ec-d24c98cae445\") " pod="e2e-gw1-7476d02544/isvc-tensorflow-runtime-predictor-bf6c5ccbc-qvlch"
Oct 30 22:30:08.037372 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:30:08.037270    4412 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/b0f81836-305f-4f94-87ec-d24c98cae445-cabundle-cert\") pod \"isvc-tensorflow-runtime-predictor-bf6c5ccbc-qvlch\" (UID: \"b0f81836-305f-4f94-87ec-d24c98cae445\") " pod="e2e-gw1-7476d02544/isvc-tensorflow-runtime-predictor-bf6c5ccbc-qvlch"
Oct 30 22:30:08.037372 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:30:08.037343    4412 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/b0f81836-305f-4f94-87ec-d24c98cae445-kserve-provision-location\") pod \"isvc-tensorflow-runtime-predictor-bf6c5ccbc-qvlch\" (UID: \"b0f81836-305f-4f94-87ec-d24c98cae445\") " pod="e2e-gw1-7476d02544/isvc-tensorflow-runtime-predictor-bf6c5ccbc-qvlch"
Oct 30 22:30:08.048301 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:30:08.048281    4412 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw1-7476d02544\"/\"openshift-service-ca.crt\""
Oct 30 22:30:08.048539 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:30:08.048518    4412 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw1-7476d02544\"/\"kube-root-ca.crt\""
Oct 30 22:30:08.048664 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:30:08.048555    4412 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw1-7476d02544\"/\"default-dockercfg-cxfkn\""
Oct 30 22:30:08.048664 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:30:08.048642    4412 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw1-7476d02544\"/\"isvc-tensorflow-runtime-predictor-serving-cert\""
Oct 30 22:30:08.050109 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:30:08.050086    4412 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw1-7476d02544/isvc-tensorflow-runtime-predictor-bf6c5ccbc-qvlch"]
Oct 30 22:30:08.138727 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:30:08.138696    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/b0f81836-305f-4f94-87ec-d24c98cae445-proxy-tls\") pod \"isvc-tensorflow-runtime-predictor-bf6c5ccbc-qvlch\" (UID: \"b0f81836-305f-4f94-87ec-d24c98cae445\") " pod="e2e-gw1-7476d02544/isvc-tensorflow-runtime-predictor-bf6c5ccbc-qvlch"
Oct 30 22:30:08.138890 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:30:08.138741    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7g95q\" (UniqueName: \"kubernetes.io/projected/b0f81836-305f-4f94-87ec-d24c98cae445-kube-api-access-7g95q\") pod \"isvc-tensorflow-runtime-predictor-bf6c5ccbc-qvlch\" (UID: \"b0f81836-305f-4f94-87ec-d24c98cae445\") " pod="e2e-gw1-7476d02544/isvc-tensorflow-runtime-predictor-bf6c5ccbc-qvlch"
Oct 30 22:30:08.138999 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:30:08.138978    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/b0f81836-305f-4f94-87ec-d24c98cae445-kube-rbac-proxy-sar-config\") pod \"isvc-tensorflow-runtime-predictor-bf6c5ccbc-qvlch\" (UID: \"b0f81836-305f-4f94-87ec-d24c98cae445\") " pod="e2e-gw1-7476d02544/isvc-tensorflow-runtime-predictor-bf6c5ccbc-qvlch"
Oct 30 22:30:08.139040 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:30:08.139016    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/b0f81836-305f-4f94-87ec-d24c98cae445-cabundle-cert\") pod \"isvc-tensorflow-runtime-predictor-bf6c5ccbc-qvlch\" (UID: \"b0f81836-305f-4f94-87ec-d24c98cae445\") " pod="e2e-gw1-7476d02544/isvc-tensorflow-runtime-predictor-bf6c5ccbc-qvlch"
Oct 30 22:30:08.139115 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:30:08.139095    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/b0f81836-305f-4f94-87ec-d24c98cae445-kserve-provision-location\") pod \"isvc-tensorflow-runtime-predictor-bf6c5ccbc-qvlch\" (UID: \"b0f81836-305f-4f94-87ec-d24c98cae445\") " pod="e2e-gw1-7476d02544/isvc-tensorflow-runtime-predictor-bf6c5ccbc-qvlch"
Oct 30 22:30:08.139481 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:30:08.139459    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/b0f81836-305f-4f94-87ec-d24c98cae445-kserve-provision-location\") pod \"isvc-tensorflow-runtime-predictor-bf6c5ccbc-qvlch\" (UID: \"b0f81836-305f-4f94-87ec-d24c98cae445\") " pod="e2e-gw1-7476d02544/isvc-tensorflow-runtime-predictor-bf6c5ccbc-qvlch"
Oct 30 22:30:08.139674 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:30:08.139657    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/b0f81836-305f-4f94-87ec-d24c98cae445-cabundle-cert\") pod \"isvc-tensorflow-runtime-predictor-bf6c5ccbc-qvlch\" (UID: \"b0f81836-305f-4f94-87ec-d24c98cae445\") " pod="e2e-gw1-7476d02544/isvc-tensorflow-runtime-predictor-bf6c5ccbc-qvlch"
Oct 30 22:30:08.139717 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:30:08.139674    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/b0f81836-305f-4f94-87ec-d24c98cae445-kube-rbac-proxy-sar-config\") pod \"isvc-tensorflow-runtime-predictor-bf6c5ccbc-qvlch\" (UID: \"b0f81836-305f-4f94-87ec-d24c98cae445\") " pod="e2e-gw1-7476d02544/isvc-tensorflow-runtime-predictor-bf6c5ccbc-qvlch"
Oct 30 22:30:08.141499 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:30:08.141472    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/b0f81836-305f-4f94-87ec-d24c98cae445-proxy-tls\") pod \"isvc-tensorflow-runtime-predictor-bf6c5ccbc-qvlch\" (UID: \"b0f81836-305f-4f94-87ec-d24c98cae445\") " pod="e2e-gw1-7476d02544/isvc-tensorflow-runtime-predictor-bf6c5ccbc-qvlch"
Oct 30 22:30:08.196798 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:30:08.196743    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7g95q\" (UniqueName: \"kubernetes.io/projected/b0f81836-305f-4f94-87ec-d24c98cae445-kube-api-access-7g95q\") pod \"isvc-tensorflow-runtime-predictor-bf6c5ccbc-qvlch\" (UID: \"b0f81836-305f-4f94-87ec-d24c98cae445\") " pod="e2e-gw1-7476d02544/isvc-tensorflow-runtime-predictor-bf6c5ccbc-qvlch"
Oct 30 22:30:08.257473 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:30:08.257444    4412 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-7476d02544/isvc-tensorflow-runtime-predictor-bf6c5ccbc-qvlch"
Oct 30 22:30:08.687031 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:30:08.687005    4412 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw1-7476d02544/isvc-tensorflow-runtime-predictor-bf6c5ccbc-qvlch"]
Oct 30 22:30:08.689467 ip-10-0-131-202 kubenswrapper[4412]: W1030 22:30:08.689438    4412 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb0f81836_305f_4f94_87ec_d24c98cae445.slice/crio-8f0980a94175bec8c4dd35cb8c4bbb43e881e031877ee44300e251c006d993c7 WatchSource:0}: Error finding container 8f0980a94175bec8c4dd35cb8c4bbb43e881e031877ee44300e251c006d993c7: Status 404 returned error can't find the container with id 8f0980a94175bec8c4dd35cb8c4bbb43e881e031877ee44300e251c006d993c7
Oct 30 22:30:09.695024 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:30:09.694983    4412 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-7476d02544/isvc-tensorflow-runtime-predictor-bf6c5ccbc-qvlch" event={"ID":"b0f81836-305f-4f94-87ec-d24c98cae445","Type":"ContainerStarted","Data":"b4dc4fcfc5989aab740189621bc1a03b507dd39d9cce0dd466969baad9f26891"}
Oct 30 22:30:09.695420 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:30:09.695037    4412 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-7476d02544/isvc-tensorflow-runtime-predictor-bf6c5ccbc-qvlch" event={"ID":"b0f81836-305f-4f94-87ec-d24c98cae445","Type":"ContainerStarted","Data":"8f0980a94175bec8c4dd35cb8c4bbb43e881e031877ee44300e251c006d993c7"}
Oct 30 22:30:14.716081 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:30:14.716052    4412 generic.go:356] "Generic (PLEG): container finished" podID="b0f81836-305f-4f94-87ec-d24c98cae445" containerID="b4dc4fcfc5989aab740189621bc1a03b507dd39d9cce0dd466969baad9f26891" exitCode=0
Oct 30 22:30:14.716525 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:30:14.716135    4412 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-7476d02544/isvc-tensorflow-runtime-predictor-bf6c5ccbc-qvlch" event={"ID":"b0f81836-305f-4f94-87ec-d24c98cae445","Type":"ContainerDied","Data":"b4dc4fcfc5989aab740189621bc1a03b507dd39d9cce0dd466969baad9f26891"}
Oct 30 22:30:15.721933 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:30:15.721901    4412 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-7476d02544/isvc-tensorflow-runtime-predictor-bf6c5ccbc-qvlch" event={"ID":"b0f81836-305f-4f94-87ec-d24c98cae445","Type":"ContainerStarted","Data":"3c9908ce0a015376437e00d9ca8b409c534dbb3eaa4bcdbc42e9e8b9d4d02114"}
Oct 30 22:30:15.722329 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:30:15.721961    4412 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-7476d02544/isvc-tensorflow-runtime-predictor-bf6c5ccbc-qvlch" event={"ID":"b0f81836-305f-4f94-87ec-d24c98cae445","Type":"ContainerStarted","Data":"f29d88ad07f0ccb7d5f65273575e76c73c534f0a313d2e15f0250b145f7d15d6"}
Oct 30 22:30:15.722329 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:30:15.722236    4412 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw1-7476d02544/isvc-tensorflow-runtime-predictor-bf6c5ccbc-qvlch"
Oct 30 22:30:15.722413 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:30:15.722398    4412 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw1-7476d02544/isvc-tensorflow-runtime-predictor-bf6c5ccbc-qvlch"
Oct 30 22:30:15.723653 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:30:15.723627    4412 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-7476d02544/isvc-tensorflow-runtime-predictor-bf6c5ccbc-qvlch" podUID="b0f81836-305f-4f94-87ec-d24c98cae445" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.25:8080: connect: connection refused"
Oct 30 22:30:15.925884 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:30:15.925834    4412 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-gw1-7476d02544/isvc-tensorflow-runtime-predictor-bf6c5ccbc-qvlch" podStartSLOduration=3.341544722 podStartE2EDuration="8.925820716s" totalImagesPullingTime="0s" totalInitContainerRuntime="5.584275994s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:30:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:30:15.891986722 +0000 UTC m=+2236.396658710" watchObservedRunningTime="2026-10-30 22:30:15.925820716 +0000 UTC m=+2236.430492736"
Oct 30 22:30:16.725510 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:30:16.725470    4412 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-7476d02544/isvc-tensorflow-runtime-predictor-bf6c5ccbc-qvlch" podUID="b0f81836-305f-4f94-87ec-d24c98cae445" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.25:8080: connect: connection refused"
Oct 30 22:30:21.730237 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:30:21.730207    4412 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw1-7476d02544/isvc-tensorflow-runtime-predictor-bf6c5ccbc-qvlch"
Oct 30 22:30:21.730716 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:30:21.730689    4412 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-7476d02544/isvc-tensorflow-runtime-predictor-bf6c5ccbc-qvlch" podUID="b0f81836-305f-4f94-87ec-d24c98cae445" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.25:8080: connect: connection refused"
Oct 30 22:30:31.730874 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:30:31.730845    4412 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw1-7476d02544/isvc-tensorflow-runtime-predictor-bf6c5ccbc-qvlch"
Oct 30 22:30:34.449291 ip-10-0-131-202 kubenswrapper[4412]: E1030 22:30:34.449247    4412 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bkhmz" containerName="extractor"
Oct 30 22:30:48.499561 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:30:48.499527    4412 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw1-7476d02544/isvc-tensorflow-runtime-predictor-bf6c5ccbc-qvlch"]
Oct 30 22:30:48.500141 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:30:48.499745    4412 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw1-7476d02544/isvc-tensorflow-runtime-predictor-bf6c5ccbc-qvlch" podUID="b0f81836-305f-4f94-87ec-d24c98cae445" containerName="kserve-container" containerID="cri-o://f29d88ad07f0ccb7d5f65273575e76c73c534f0a313d2e15f0250b145f7d15d6" gracePeriod=30
Oct 30 22:30:48.500141 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:30:48.499808    4412 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw1-7476d02544/isvc-tensorflow-runtime-predictor-bf6c5ccbc-qvlch" podUID="b0f81836-305f-4f94-87ec-d24c98cae445" containerName="kube-rbac-proxy" containerID="cri-o://3c9908ce0a015376437e00d9ca8b409c534dbb3eaa4bcdbc42e9e8b9d4d02114" gracePeriod=30
Oct 30 22:30:48.844407 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:30:48.844320    4412 generic.go:356] "Generic (PLEG): container finished" podID="b0f81836-305f-4f94-87ec-d24c98cae445" containerID="3c9908ce0a015376437e00d9ca8b409c534dbb3eaa4bcdbc42e9e8b9d4d02114" exitCode=0
Oct 30 22:30:48.844407 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:30:48.844358    4412 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-7476d02544/isvc-tensorflow-runtime-predictor-bf6c5ccbc-qvlch" event={"ID":"b0f81836-305f-4f94-87ec-d24c98cae445","Type":"ContainerDied","Data":"3c9908ce0a015376437e00d9ca8b409c534dbb3eaa4bcdbc42e9e8b9d4d02114"}
Oct 30 22:30:51.726375 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:30:51.726334    4412 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-7476d02544/isvc-tensorflow-runtime-predictor-bf6c5ccbc-qvlch" podUID="b0f81836-305f-4f94-87ec-d24c98cae445" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.25:8643/healthz\": dial tcp 10.133.0.25:8643: connect: connection refused"
Oct 30 22:30:56.725931 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:30:56.725891    4412 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-7476d02544/isvc-tensorflow-runtime-predictor-bf6c5ccbc-qvlch" podUID="b0f81836-305f-4f94-87ec-d24c98cae445" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.25:8643/healthz\": dial tcp 10.133.0.25:8643: connect: connection refused"
Oct 30 22:31:01.726454 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:31:01.726399    4412 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-7476d02544/isvc-tensorflow-runtime-predictor-bf6c5ccbc-qvlch" podUID="b0f81836-305f-4f94-87ec-d24c98cae445" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.25:8643/healthz\": dial tcp 10.133.0.25:8643: connect: connection refused"
Oct 30 22:31:01.726906 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:31:01.726562    4412 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw1-7476d02544/isvc-tensorflow-runtime-predictor-bf6c5ccbc-qvlch"
Oct 30 22:31:04.447860 ip-10-0-131-202 kubenswrapper[4412]: E1030 22:31:04.447821    4412 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bkhmz" containerName="extractor"
Oct 30 22:31:06.726233 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:31:06.726189    4412 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-7476d02544/isvc-tensorflow-runtime-predictor-bf6c5ccbc-qvlch" podUID="b0f81836-305f-4f94-87ec-d24c98cae445" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.25:8643/healthz\": dial tcp 10.133.0.25:8643: connect: connection refused"
Oct 30 22:31:11.726672 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:31:11.726580    4412 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-7476d02544/isvc-tensorflow-runtime-predictor-bf6c5ccbc-qvlch" podUID="b0f81836-305f-4f94-87ec-d24c98cae445" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.25:8643/healthz\": dial tcp 10.133.0.25:8643: connect: connection refused"
Oct 30 22:31:16.726006 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:31:16.725965    4412 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-7476d02544/isvc-tensorflow-runtime-predictor-bf6c5ccbc-qvlch" podUID="b0f81836-305f-4f94-87ec-d24c98cae445" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.25:8643/healthz\": dial tcp 10.133.0.25:8643: connect: connection refused"
Oct 30 22:31:18.959428 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:31:18.959345    4412 generic.go:356] "Generic (PLEG): container finished" podID="b0f81836-305f-4f94-87ec-d24c98cae445" containerID="f29d88ad07f0ccb7d5f65273575e76c73c534f0a313d2e15f0250b145f7d15d6" exitCode=137
Oct 30 22:31:18.959428 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:31:18.959396    4412 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-7476d02544/isvc-tensorflow-runtime-predictor-bf6c5ccbc-qvlch" event={"ID":"b0f81836-305f-4f94-87ec-d24c98cae445","Type":"ContainerDied","Data":"f29d88ad07f0ccb7d5f65273575e76c73c534f0a313d2e15f0250b145f7d15d6"}
Oct 30 22:31:19.140304 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:31:19.140274    4412 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-7476d02544/isvc-tensorflow-runtime-predictor-bf6c5ccbc-qvlch"
Oct 30 22:31:19.206852 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:31:19.206818    4412 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b0f81836-305f-4f94-87ec-d24c98cae445-kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/b0f81836-305f-4f94-87ec-d24c98cae445-kserve-provision-location\") pod \"b0f81836-305f-4f94-87ec-d24c98cae445\" (UID: \"b0f81836-305f-4f94-87ec-d24c98cae445\") "
Oct 30 22:31:19.207161 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:31:19.206873    4412 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b0f81836-305f-4f94-87ec-d24c98cae445-kube-api-access-7g95q\" (UniqueName: \"kubernetes.io/projected/b0f81836-305f-4f94-87ec-d24c98cae445-kube-api-access-7g95q\") pod \"b0f81836-305f-4f94-87ec-d24c98cae445\" (UID: \"b0f81836-305f-4f94-87ec-d24c98cae445\") "
Oct 30 22:31:19.207161 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:31:19.206925    4412 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b0f81836-305f-4f94-87ec-d24c98cae445-kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/b0f81836-305f-4f94-87ec-d24c98cae445-kube-rbac-proxy-sar-config\") pod \"b0f81836-305f-4f94-87ec-d24c98cae445\" (UID: \"b0f81836-305f-4f94-87ec-d24c98cae445\") "
Oct 30 22:31:19.207161 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:31:19.207000    4412 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/b0f81836-305f-4f94-87ec-d24c98cae445-proxy-tls\" (UniqueName: \"kubernetes.io/secret/b0f81836-305f-4f94-87ec-d24c98cae445-proxy-tls\") pod \"b0f81836-305f-4f94-87ec-d24c98cae445\" (UID: \"b0f81836-305f-4f94-87ec-d24c98cae445\") "
Oct 30 22:31:19.207161 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:31:19.207047    4412 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b0f81836-305f-4f94-87ec-d24c98cae445-cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/b0f81836-305f-4f94-87ec-d24c98cae445-cabundle-cert\") pod \"b0f81836-305f-4f94-87ec-d24c98cae445\" (UID: \"b0f81836-305f-4f94-87ec-d24c98cae445\") "
Oct 30 22:31:19.207348 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:31:19.207278    4412 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b0f81836-305f-4f94-87ec-d24c98cae445-kube-rbac-proxy-sar-config" pod "b0f81836-305f-4f94-87ec-d24c98cae445" (UID: "b0f81836-305f-4f94-87ec-d24c98cae445"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:31:19.207541 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:31:19.207509    4412 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b0f81836-305f-4f94-87ec-d24c98cae445-cabundle-cert" pod "b0f81836-305f-4f94-87ec-d24c98cae445" (UID: "b0f81836-305f-4f94-87ec-d24c98cae445"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:31:19.209190 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:31:19.209164    4412 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b0f81836-305f-4f94-87ec-d24c98cae445-kube-api-access-7g95q" pod "b0f81836-305f-4f94-87ec-d24c98cae445" (UID: "b0f81836-305f-4f94-87ec-d24c98cae445"). InnerVolumeSpecName "kube-api-access-7g95q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:31:19.209281 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:31:19.209187    4412 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b0f81836-305f-4f94-87ec-d24c98cae445-proxy-tls" pod "b0f81836-305f-4f94-87ec-d24c98cae445" (UID: "b0f81836-305f-4f94-87ec-d24c98cae445"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:31:19.217449 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:31:19.217427    4412 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b0f81836-305f-4f94-87ec-d24c98cae445-kserve-provision-location" pod "b0f81836-305f-4f94-87ec-d24c98cae445" (UID: "b0f81836-305f-4f94-87ec-d24c98cae445"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:31:19.308012 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:31:19.307977    4412 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7g95q\" (UniqueName: \"kubernetes.io/projected/b0f81836-305f-4f94-87ec-d24c98cae445-kube-api-access-7g95q\") on node \"ip-10-0-131-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:31:19.308012 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:31:19.308009    4412 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/b0f81836-305f-4f94-87ec-d24c98cae445-kube-rbac-proxy-sar-config\") on node \"ip-10-0-131-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:31:19.308012 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:31:19.308020    4412 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/b0f81836-305f-4f94-87ec-d24c98cae445-proxy-tls\") on node \"ip-10-0-131-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:31:19.308248 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:31:19.308029    4412 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/b0f81836-305f-4f94-87ec-d24c98cae445-cabundle-cert\") on node \"ip-10-0-131-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:31:19.308248 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:31:19.308038    4412 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/b0f81836-305f-4f94-87ec-d24c98cae445-kserve-provision-location\") on node \"ip-10-0-131-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:31:19.965395 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:31:19.965361    4412 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-7476d02544/isvc-tensorflow-runtime-predictor-bf6c5ccbc-qvlch" event={"ID":"b0f81836-305f-4f94-87ec-d24c98cae445","Type":"ContainerDied","Data":"8f0980a94175bec8c4dd35cb8c4bbb43e881e031877ee44300e251c006d993c7"}
Oct 30 22:31:19.965860 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:31:19.965411    4412 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-7476d02544/isvc-tensorflow-runtime-predictor-bf6c5ccbc-qvlch"
Oct 30 22:31:19.965860 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:31:19.965428    4412 scope.go:122] "RemoveContainer" containerID="3c9908ce0a015376437e00d9ca8b409c534dbb3eaa4bcdbc42e9e8b9d4d02114"
Oct 30 22:31:19.973522 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:31:19.973498    4412 scope.go:122] "RemoveContainer" containerID="f29d88ad07f0ccb7d5f65273575e76c73c534f0a313d2e15f0250b145f7d15d6"
Oct 30 22:31:19.980447 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:31:19.980428    4412 scope.go:122] "RemoveContainer" containerID="b4dc4fcfc5989aab740189621bc1a03b507dd39d9cce0dd466969baad9f26891"
Oct 30 22:31:20.181397 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:31:20.181366    4412 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw1-7476d02544/isvc-tensorflow-runtime-predictor-bf6c5ccbc-qvlch"]
Oct 30 22:31:20.231764 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:31:20.231691    4412 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-gw1-7476d02544/isvc-tensorflow-runtime-predictor-bf6c5ccbc-qvlch"]
Oct 30 22:31:21.625975 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:31:21.625940    4412 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b0f81836-305f-4f94-87ec-d24c98cae445" path="/var/lib/kubelet/pods/b0f81836-305f-4f94-87ec-d24c98cae445/volumes"
Oct 30 22:31:24.027464 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:31:24.027435    4412 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-gw1-7476d02544/isvc-triton-predictor-795d5b7fd4-945xn"]
Oct 30 22:31:24.027871 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:31:24.027717    4412 cpu_manager.go:414] "containerMap: removing container" podUID="b0f81836-305f-4f94-87ec-d24c98cae445" containerName="kserve-container"
Oct 30 22:31:24.027871 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:31:24.027728    4412 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b0f81836-305f-4f94-87ec-d24c98cae445" containerName="kserve-container"
Oct 30 22:31:24.027871 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:31:24.027741    4412 cpu_manager.go:414] "containerMap: removing container" podUID="b0f81836-305f-4f94-87ec-d24c98cae445" containerName="storage-initializer"
Oct 30 22:31:24.027871 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:31:24.027746    4412 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b0f81836-305f-4f94-87ec-d24c98cae445" containerName="storage-initializer"
Oct 30 22:31:24.027871 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:31:24.027763    4412 cpu_manager.go:414] "containerMap: removing container" podUID="b0f81836-305f-4f94-87ec-d24c98cae445" containerName="kube-rbac-proxy"
Oct 30 22:31:24.027871 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:31:24.027767    4412 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b0f81836-305f-4f94-87ec-d24c98cae445" containerName="kube-rbac-proxy"
Oct 30 22:31:24.027871 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:31:24.027811    4412 memory_manager.go:365] "RemoveStaleState removing state" podUID="b0f81836-305f-4f94-87ec-d24c98cae445" containerName="kube-rbac-proxy"
Oct 30 22:31:24.027871 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:31:24.027821    4412 memory_manager.go:365] "RemoveStaleState removing state" podUID="b0f81836-305f-4f94-87ec-d24c98cae445" containerName="kserve-container"
Oct 30 22:31:24.032443 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:31:24.032427    4412 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-7476d02544/isvc-triton-predictor-795d5b7fd4-945xn"
Oct 30 22:31:24.085907 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:31:24.085885    4412 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw1-7476d02544\"/\"isvc-triton-kube-rbac-proxy-sar-config\""
Oct 30 22:31:24.085907 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:31:24.085898    4412 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw1-7476d02544\"/\"openshift-service-ca.crt\""
Oct 30 22:31:24.085907 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:31:24.085886    4412 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw1-7476d02544\"/\"odh-kserve-custom-ca-bundle\""
Oct 30 22:31:24.086110 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:31:24.085935    4412 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw1-7476d02544\"/\"seaweedfs-s3-creds\""
Oct 30 22:31:24.136150 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:31:24.136106    4412 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw1-7476d02544\"/\"kube-root-ca.crt\""
Oct 30 22:31:24.136269 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:31:24.136199    4412 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw1-7476d02544\"/\"isvc-triton-predictor-serving-cert\""
Oct 30 22:31:24.136269 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:31:24.136250    4412 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw1-7476d02544\"/\"default-dockercfg-cxfkn\""
Oct 30 22:31:24.137608 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:31:24.137592    4412 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw1-7476d02544/isvc-triton-predictor-795d5b7fd4-945xn"]
Oct 30 22:31:24.146741 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:31:24.146718    4412 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/4cde3f35-4b9b-48c9-b03b-9888e545665b-proxy-tls\") pod \"isvc-triton-predictor-795d5b7fd4-945xn\" (UID: \"4cde3f35-4b9b-48c9-b03b-9888e545665b\") " pod="e2e-gw1-7476d02544/isvc-triton-predictor-795d5b7fd4-945xn"
Oct 30 22:31:24.146823 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:31:24.146752    4412 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/4cde3f35-4b9b-48c9-b03b-9888e545665b-kserve-provision-location\") pod \"isvc-triton-predictor-795d5b7fd4-945xn\" (UID: \"4cde3f35-4b9b-48c9-b03b-9888e545665b\") " pod="e2e-gw1-7476d02544/isvc-triton-predictor-795d5b7fd4-945xn"
Oct 30 22:31:24.146823 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:31:24.146780    4412 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/4cde3f35-4b9b-48c9-b03b-9888e545665b-cabundle-cert\") pod \"isvc-triton-predictor-795d5b7fd4-945xn\" (UID: \"4cde3f35-4b9b-48c9-b03b-9888e545665b\") " pod="e2e-gw1-7476d02544/isvc-triton-predictor-795d5b7fd4-945xn"
Oct 30 22:31:24.146939 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:31:24.146915    4412 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9t4nz\" (UniqueName: \"kubernetes.io/projected/4cde3f35-4b9b-48c9-b03b-9888e545665b-kube-api-access-9t4nz\") pod \"isvc-triton-predictor-795d5b7fd4-945xn\" (UID: \"4cde3f35-4b9b-48c9-b03b-9888e545665b\") " pod="e2e-gw1-7476d02544/isvc-triton-predictor-795d5b7fd4-945xn"
Oct 30 22:31:24.146974 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:31:24.146953    4412 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/4cde3f35-4b9b-48c9-b03b-9888e545665b-kube-rbac-proxy-sar-config\") pod \"isvc-triton-predictor-795d5b7fd4-945xn\" (UID: \"4cde3f35-4b9b-48c9-b03b-9888e545665b\") " pod="e2e-gw1-7476d02544/isvc-triton-predictor-795d5b7fd4-945xn"
Oct 30 22:31:24.248011 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:31:24.247977    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/4cde3f35-4b9b-48c9-b03b-9888e545665b-cabundle-cert\") pod \"isvc-triton-predictor-795d5b7fd4-945xn\" (UID: \"4cde3f35-4b9b-48c9-b03b-9888e545665b\") " pod="e2e-gw1-7476d02544/isvc-triton-predictor-795d5b7fd4-945xn"
Oct 30 22:31:24.248181 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:31:24.248034    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9t4nz\" (UniqueName: \"kubernetes.io/projected/4cde3f35-4b9b-48c9-b03b-9888e545665b-kube-api-access-9t4nz\") pod \"isvc-triton-predictor-795d5b7fd4-945xn\" (UID: \"4cde3f35-4b9b-48c9-b03b-9888e545665b\") " pod="e2e-gw1-7476d02544/isvc-triton-predictor-795d5b7fd4-945xn"
Oct 30 22:31:24.248181 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:31:24.248061    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/4cde3f35-4b9b-48c9-b03b-9888e545665b-kube-rbac-proxy-sar-config\") pod \"isvc-triton-predictor-795d5b7fd4-945xn\" (UID: \"4cde3f35-4b9b-48c9-b03b-9888e545665b\") " pod="e2e-gw1-7476d02544/isvc-triton-predictor-795d5b7fd4-945xn"
Oct 30 22:31:24.248181 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:31:24.248138    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/4cde3f35-4b9b-48c9-b03b-9888e545665b-proxy-tls\") pod \"isvc-triton-predictor-795d5b7fd4-945xn\" (UID: \"4cde3f35-4b9b-48c9-b03b-9888e545665b\") " pod="e2e-gw1-7476d02544/isvc-triton-predictor-795d5b7fd4-945xn"
Oct 30 22:31:24.248344 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:31:24.248317    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/4cde3f35-4b9b-48c9-b03b-9888e545665b-kserve-provision-location\") pod \"isvc-triton-predictor-795d5b7fd4-945xn\" (UID: \"4cde3f35-4b9b-48c9-b03b-9888e545665b\") " pod="e2e-gw1-7476d02544/isvc-triton-predictor-795d5b7fd4-945xn"
Oct 30 22:31:24.248673 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:31:24.248646    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/4cde3f35-4b9b-48c9-b03b-9888e545665b-kserve-provision-location\") pod \"isvc-triton-predictor-795d5b7fd4-945xn\" (UID: \"4cde3f35-4b9b-48c9-b03b-9888e545665b\") " pod="e2e-gw1-7476d02544/isvc-triton-predictor-795d5b7fd4-945xn"
Oct 30 22:31:24.248766 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:31:24.248752    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/4cde3f35-4b9b-48c9-b03b-9888e545665b-kube-rbac-proxy-sar-config\") pod \"isvc-triton-predictor-795d5b7fd4-945xn\" (UID: \"4cde3f35-4b9b-48c9-b03b-9888e545665b\") " pod="e2e-gw1-7476d02544/isvc-triton-predictor-795d5b7fd4-945xn"
Oct 30 22:31:24.248811 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:31:24.248781    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/4cde3f35-4b9b-48c9-b03b-9888e545665b-cabundle-cert\") pod \"isvc-triton-predictor-795d5b7fd4-945xn\" (UID: \"4cde3f35-4b9b-48c9-b03b-9888e545665b\") " pod="e2e-gw1-7476d02544/isvc-triton-predictor-795d5b7fd4-945xn"
Oct 30 22:31:24.250606 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:31:24.250584    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/4cde3f35-4b9b-48c9-b03b-9888e545665b-proxy-tls\") pod \"isvc-triton-predictor-795d5b7fd4-945xn\" (UID: \"4cde3f35-4b9b-48c9-b03b-9888e545665b\") " pod="e2e-gw1-7476d02544/isvc-triton-predictor-795d5b7fd4-945xn"
Oct 30 22:31:24.307857 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:31:24.307795    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9t4nz\" (UniqueName: \"kubernetes.io/projected/4cde3f35-4b9b-48c9-b03b-9888e545665b-kube-api-access-9t4nz\") pod \"isvc-triton-predictor-795d5b7fd4-945xn\" (UID: \"4cde3f35-4b9b-48c9-b03b-9888e545665b\") " pod="e2e-gw1-7476d02544/isvc-triton-predictor-795d5b7fd4-945xn"
Oct 30 22:31:24.348876 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:31:24.348847    4412 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-7476d02544/isvc-triton-predictor-795d5b7fd4-945xn"
Oct 30 22:31:24.783618 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:31:24.783591    4412 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw1-7476d02544/isvc-triton-predictor-795d5b7fd4-945xn"]
Oct 30 22:31:24.785635 ip-10-0-131-202 kubenswrapper[4412]: W1030 22:31:24.785611    4412 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4cde3f35_4b9b_48c9_b03b_9888e545665b.slice/crio-0aa25bc5a3775caa00417b137b03fee38d34f478453956b53cddacdeb93d192b WatchSource:0}: Error finding container 0aa25bc5a3775caa00417b137b03fee38d34f478453956b53cddacdeb93d192b: Status 404 returned error can't find the container with id 0aa25bc5a3775caa00417b137b03fee38d34f478453956b53cddacdeb93d192b
Oct 30 22:31:24.985583 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:31:24.985546    4412 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-7476d02544/isvc-triton-predictor-795d5b7fd4-945xn" event={"ID":"4cde3f35-4b9b-48c9-b03b-9888e545665b","Type":"ContainerStarted","Data":"32888e3556f93e5321d8eddcba2b215edb92a41eb1957d48da6d829fa3827e52"}
Oct 30 22:31:24.985783 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:31:24.985610    4412 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-7476d02544/isvc-triton-predictor-795d5b7fd4-945xn" event={"ID":"4cde3f35-4b9b-48c9-b03b-9888e545665b","Type":"ContainerStarted","Data":"0aa25bc5a3775caa00417b137b03fee38d34f478453956b53cddacdeb93d192b"}
Oct 30 22:31:29.003399 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:31:29.003364    4412 generic.go:356] "Generic (PLEG): container finished" podID="4cde3f35-4b9b-48c9-b03b-9888e545665b" containerID="32888e3556f93e5321d8eddcba2b215edb92a41eb1957d48da6d829fa3827e52" exitCode=0
Oct 30 22:31:29.003889 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:31:29.003445    4412 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-7476d02544/isvc-triton-predictor-795d5b7fd4-945xn" event={"ID":"4cde3f35-4b9b-48c9-b03b-9888e545665b","Type":"ContainerDied","Data":"32888e3556f93e5321d8eddcba2b215edb92a41eb1957d48da6d829fa3827e52"}
Oct 30 22:31:29.004741 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:31:29.004724    4412 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 22:31:34.467971 ip-10-0-131-202 kubenswrapper[4412]: E1030 22:31:34.467929    4412 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bkhmz" containerName="extractor"
Oct 30 22:32:04.454628 ip-10-0-131-202 kubenswrapper[4412]: E1030 22:32:04.454521    4412 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bkhmz" containerName="extractor"
Oct 30 22:32:34.452911 ip-10-0-131-202 kubenswrapper[4412]: E1030 22:32:34.452874    4412 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bkhmz" containerName="extractor"
Oct 30 22:33:04.581630 ip-10-0-131-202 kubenswrapper[4412]: E1030 22:33:04.581460    4412 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bkhmz" containerName="extractor"
Oct 30 22:33:23.539112 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:33:23.539072    4412 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-7476d02544/isvc-triton-predictor-795d5b7fd4-945xn" event={"ID":"4cde3f35-4b9b-48c9-b03b-9888e545665b","Type":"ContainerStarted","Data":"2b3bd85f33981fe7baa7fe17c2792954c2cdf2c5699c6493b72872dcb4dbfc77"}
Oct 30 22:33:23.539594 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:33:23.539153    4412 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-7476d02544/isvc-triton-predictor-795d5b7fd4-945xn" event={"ID":"4cde3f35-4b9b-48c9-b03b-9888e545665b","Type":"ContainerStarted","Data":"e80b36e55da6409d9530fc0061ef6e7c3fa225887590557a5cd648d934aac102"}
Oct 30 22:33:23.539594 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:33:23.539391    4412 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw1-7476d02544/isvc-triton-predictor-795d5b7fd4-945xn"
Oct 30 22:33:23.539594 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:33:23.539508    4412 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw1-7476d02544/isvc-triton-predictor-795d5b7fd4-945xn"
Oct 30 22:33:23.540954 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:33:23.540929    4412 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-7476d02544/isvc-triton-predictor-795d5b7fd4-945xn" podUID="4cde3f35-4b9b-48c9-b03b-9888e545665b" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.26:8080: connect: connection refused"
Oct 30 22:33:23.660451 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:33:23.660401    4412 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-gw1-7476d02544/isvc-triton-predictor-795d5b7fd4-945xn" podStartSLOduration=2.600142529 podStartE2EDuration="2m0.66038869s" totalImagesPullingTime="1m54.132442012s" totalInitContainerRuntime="3.927804149s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:31:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:33:23.659400165 +0000 UTC m=+2424.164072130" watchObservedRunningTime="2026-10-30 22:33:23.66038869 +0000 UTC m=+2424.165060714"
Oct 30 22:33:24.543542 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:33:24.543510    4412 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-7476d02544/isvc-triton-predictor-795d5b7fd4-945xn" podUID="4cde3f35-4b9b-48c9-b03b-9888e545665b" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.26:8080: connect: connection refused"
Oct 30 22:33:29.548959 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:33:29.548930    4412 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw1-7476d02544/isvc-triton-predictor-795d5b7fd4-945xn"
Oct 30 22:33:29.549500 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:33:29.549473    4412 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-7476d02544/isvc-triton-predictor-795d5b7fd4-945xn" podUID="4cde3f35-4b9b-48c9-b03b-9888e545665b" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.26:8080: connect: connection refused"
Oct 30 22:33:34.447480 ip-10-0-131-202 kubenswrapper[4412]: E1030 22:33:34.447440    4412 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bkhmz" containerName="extractor"
Oct 30 22:33:39.550035 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:33:39.550007    4412 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw1-7476d02544/isvc-triton-predictor-795d5b7fd4-945xn"
Oct 30 22:33:46.478935 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:33:46.478906    4412 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw1-7476d02544/isvc-triton-predictor-795d5b7fd4-945xn"]
Oct 30 22:33:46.479352 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:33:46.479142    4412 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw1-7476d02544/isvc-triton-predictor-795d5b7fd4-945xn" podUID="4cde3f35-4b9b-48c9-b03b-9888e545665b" containerName="kserve-container" containerID="cri-o://e80b36e55da6409d9530fc0061ef6e7c3fa225887590557a5cd648d934aac102" gracePeriod=30
Oct 30 22:33:46.479352 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:33:46.479196    4412 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw1-7476d02544/isvc-triton-predictor-795d5b7fd4-945xn" podUID="4cde3f35-4b9b-48c9-b03b-9888e545665b" containerName="kube-rbac-proxy" containerID="cri-o://2b3bd85f33981fe7baa7fe17c2792954c2cdf2c5699c6493b72872dcb4dbfc77" gracePeriod=30
Oct 30 22:33:46.625986 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:33:46.625958    4412 generic.go:356] "Generic (PLEG): container finished" podID="4cde3f35-4b9b-48c9-b03b-9888e545665b" containerID="2b3bd85f33981fe7baa7fe17c2792954c2cdf2c5699c6493b72872dcb4dbfc77" exitCode=0
Oct 30 22:33:46.626096 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:33:46.626028    4412 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-7476d02544/isvc-triton-predictor-795d5b7fd4-945xn" event={"ID":"4cde3f35-4b9b-48c9-b03b-9888e545665b","Type":"ContainerDied","Data":"2b3bd85f33981fe7baa7fe17c2792954c2cdf2c5699c6493b72872dcb4dbfc77"}
Oct 30 22:33:49.059739 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:33:49.059715    4412 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-7476d02544/isvc-triton-predictor-795d5b7fd4-945xn"
Oct 30 22:33:49.118186 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:33:49.118163    4412 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4cde3f35-4b9b-48c9-b03b-9888e545665b-kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/4cde3f35-4b9b-48c9-b03b-9888e545665b-kserve-provision-location\") pod \"4cde3f35-4b9b-48c9-b03b-9888e545665b\" (UID: \"4cde3f35-4b9b-48c9-b03b-9888e545665b\") "
Oct 30 22:33:49.118317 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:33:49.118214    4412 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/4cde3f35-4b9b-48c9-b03b-9888e545665b-cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/4cde3f35-4b9b-48c9-b03b-9888e545665b-cabundle-cert\") pod \"4cde3f35-4b9b-48c9-b03b-9888e545665b\" (UID: \"4cde3f35-4b9b-48c9-b03b-9888e545665b\") "
Oct 30 22:33:49.118317 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:33:49.118239    4412 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/4cde3f35-4b9b-48c9-b03b-9888e545665b-proxy-tls\" (UniqueName: \"kubernetes.io/secret/4cde3f35-4b9b-48c9-b03b-9888e545665b-proxy-tls\") pod \"4cde3f35-4b9b-48c9-b03b-9888e545665b\" (UID: \"4cde3f35-4b9b-48c9-b03b-9888e545665b\") "
Oct 30 22:33:49.118404 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:33:49.118363    4412 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/4cde3f35-4b9b-48c9-b03b-9888e545665b-kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/4cde3f35-4b9b-48c9-b03b-9888e545665b-kube-rbac-proxy-sar-config\") pod \"4cde3f35-4b9b-48c9-b03b-9888e545665b\" (UID: \"4cde3f35-4b9b-48c9-b03b-9888e545665b\") "
Oct 30 22:33:49.118467 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:33:49.118401    4412 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4cde3f35-4b9b-48c9-b03b-9888e545665b-kube-api-access-9t4nz\" (UniqueName: \"kubernetes.io/projected/4cde3f35-4b9b-48c9-b03b-9888e545665b-kube-api-access-9t4nz\") pod \"4cde3f35-4b9b-48c9-b03b-9888e545665b\" (UID: \"4cde3f35-4b9b-48c9-b03b-9888e545665b\") "
Oct 30 22:33:49.118724 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:33:49.118683    4412 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4cde3f35-4b9b-48c9-b03b-9888e545665b-kserve-provision-location" pod "4cde3f35-4b9b-48c9-b03b-9888e545665b" (UID: "4cde3f35-4b9b-48c9-b03b-9888e545665b"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:33:49.118724 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:33:49.118708    4412 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4cde3f35-4b9b-48c9-b03b-9888e545665b-cabundle-cert" pod "4cde3f35-4b9b-48c9-b03b-9888e545665b" (UID: "4cde3f35-4b9b-48c9-b03b-9888e545665b"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:33:49.118894 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:33:49.118804    4412 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4cde3f35-4b9b-48c9-b03b-9888e545665b-kube-rbac-proxy-sar-config" pod "4cde3f35-4b9b-48c9-b03b-9888e545665b" (UID: "4cde3f35-4b9b-48c9-b03b-9888e545665b"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:33:49.125414 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:33:49.125390    4412 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4cde3f35-4b9b-48c9-b03b-9888e545665b-kube-api-access-9t4nz" pod "4cde3f35-4b9b-48c9-b03b-9888e545665b" (UID: "4cde3f35-4b9b-48c9-b03b-9888e545665b"). InnerVolumeSpecName "kube-api-access-9t4nz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:33:49.125414 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:33:49.125391    4412 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4cde3f35-4b9b-48c9-b03b-9888e545665b-proxy-tls" pod "4cde3f35-4b9b-48c9-b03b-9888e545665b" (UID: "4cde3f35-4b9b-48c9-b03b-9888e545665b"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:33:49.219788 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:33:49.219766    4412 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/4cde3f35-4b9b-48c9-b03b-9888e545665b-kube-rbac-proxy-sar-config\") on node \"ip-10-0-131-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:33:49.219788 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:33:49.219788    4412 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9t4nz\" (UniqueName: \"kubernetes.io/projected/4cde3f35-4b9b-48c9-b03b-9888e545665b-kube-api-access-9t4nz\") on node \"ip-10-0-131-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:33:49.219926 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:33:49.219797    4412 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/4cde3f35-4b9b-48c9-b03b-9888e545665b-kserve-provision-location\") on node \"ip-10-0-131-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:33:49.219926 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:33:49.219807    4412 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/4cde3f35-4b9b-48c9-b03b-9888e545665b-cabundle-cert\") on node \"ip-10-0-131-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:33:49.219926 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:33:49.219816    4412 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/4cde3f35-4b9b-48c9-b03b-9888e545665b-proxy-tls\") on node \"ip-10-0-131-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:33:49.638918 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:33:49.638891    4412 generic.go:356] "Generic (PLEG): container finished" podID="4cde3f35-4b9b-48c9-b03b-9888e545665b" containerID="e80b36e55da6409d9530fc0061ef6e7c3fa225887590557a5cd648d934aac102" exitCode=0
Oct 30 22:33:49.639046 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:33:49.638949    4412 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-7476d02544/isvc-triton-predictor-795d5b7fd4-945xn"
Oct 30 22:33:49.639046 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:33:49.638969    4412 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-7476d02544/isvc-triton-predictor-795d5b7fd4-945xn" event={"ID":"4cde3f35-4b9b-48c9-b03b-9888e545665b","Type":"ContainerDied","Data":"e80b36e55da6409d9530fc0061ef6e7c3fa225887590557a5cd648d934aac102"}
Oct 30 22:33:49.639046 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:33:49.639035    4412 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-7476d02544/isvc-triton-predictor-795d5b7fd4-945xn" event={"ID":"4cde3f35-4b9b-48c9-b03b-9888e545665b","Type":"ContainerDied","Data":"0aa25bc5a3775caa00417b137b03fee38d34f478453956b53cddacdeb93d192b"}
Oct 30 22:33:49.639171 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:33:49.639072    4412 scope.go:122] "RemoveContainer" containerID="2b3bd85f33981fe7baa7fe17c2792954c2cdf2c5699c6493b72872dcb4dbfc77"
Oct 30 22:33:49.647228 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:33:49.647207    4412 scope.go:122] "RemoveContainer" containerID="e80b36e55da6409d9530fc0061ef6e7c3fa225887590557a5cd648d934aac102"
Oct 30 22:33:49.654951 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:33:49.654924    4412 scope.go:122] "RemoveContainer" containerID="32888e3556f93e5321d8eddcba2b215edb92a41eb1957d48da6d829fa3827e52"
Oct 30 22:33:49.662409 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:33:49.662384    4412 scope.go:122] "RemoveContainer" containerID="2b3bd85f33981fe7baa7fe17c2792954c2cdf2c5699c6493b72872dcb4dbfc77"
Oct 30 22:33:49.662693 ip-10-0-131-202 kubenswrapper[4412]: E1030 22:33:49.662677    4412 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2b3bd85f33981fe7baa7fe17c2792954c2cdf2c5699c6493b72872dcb4dbfc77\": container with ID starting with 2b3bd85f33981fe7baa7fe17c2792954c2cdf2c5699c6493b72872dcb4dbfc77 not found: ID does not exist" containerID="2b3bd85f33981fe7baa7fe17c2792954c2cdf2c5699c6493b72872dcb4dbfc77"
Oct 30 22:33:49.662768 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:33:49.662702    4412 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2b3bd85f33981fe7baa7fe17c2792954c2cdf2c5699c6493b72872dcb4dbfc77"} err="failed to get container status \"2b3bd85f33981fe7baa7fe17c2792954c2cdf2c5699c6493b72872dcb4dbfc77\": rpc error: code = NotFound desc = could not find container \"2b3bd85f33981fe7baa7fe17c2792954c2cdf2c5699c6493b72872dcb4dbfc77\": container with ID starting with 2b3bd85f33981fe7baa7fe17c2792954c2cdf2c5699c6493b72872dcb4dbfc77 not found: ID does not exist"
Oct 30 22:33:49.662768 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:33:49.662720    4412 scope.go:122] "RemoveContainer" containerID="e80b36e55da6409d9530fc0061ef6e7c3fa225887590557a5cd648d934aac102"
Oct 30 22:33:49.662998 ip-10-0-131-202 kubenswrapper[4412]: E1030 22:33:49.662975    4412 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e80b36e55da6409d9530fc0061ef6e7c3fa225887590557a5cd648d934aac102\": container with ID starting with e80b36e55da6409d9530fc0061ef6e7c3fa225887590557a5cd648d934aac102 not found: ID does not exist" containerID="e80b36e55da6409d9530fc0061ef6e7c3fa225887590557a5cd648d934aac102"
Oct 30 22:33:49.663061 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:33:49.663005    4412 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e80b36e55da6409d9530fc0061ef6e7c3fa225887590557a5cd648d934aac102"} err="failed to get container status \"e80b36e55da6409d9530fc0061ef6e7c3fa225887590557a5cd648d934aac102\": rpc error: code = NotFound desc = could not find container \"e80b36e55da6409d9530fc0061ef6e7c3fa225887590557a5cd648d934aac102\": container with ID starting with e80b36e55da6409d9530fc0061ef6e7c3fa225887590557a5cd648d934aac102 not found: ID does not exist"
Oct 30 22:33:49.663061 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:33:49.663021    4412 scope.go:122] "RemoveContainer" containerID="32888e3556f93e5321d8eddcba2b215edb92a41eb1957d48da6d829fa3827e52"
Oct 30 22:33:49.663381 ip-10-0-131-202 kubenswrapper[4412]: E1030 22:33:49.663365    4412 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"32888e3556f93e5321d8eddcba2b215edb92a41eb1957d48da6d829fa3827e52\": container with ID starting with 32888e3556f93e5321d8eddcba2b215edb92a41eb1957d48da6d829fa3827e52 not found: ID does not exist" containerID="32888e3556f93e5321d8eddcba2b215edb92a41eb1957d48da6d829fa3827e52"
Oct 30 22:33:49.663440 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:33:49.663386    4412 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"32888e3556f93e5321d8eddcba2b215edb92a41eb1957d48da6d829fa3827e52"} err="failed to get container status \"32888e3556f93e5321d8eddcba2b215edb92a41eb1957d48da6d829fa3827e52\": rpc error: code = NotFound desc = could not find container \"32888e3556f93e5321d8eddcba2b215edb92a41eb1957d48da6d829fa3827e52\": container with ID starting with 32888e3556f93e5321d8eddcba2b215edb92a41eb1957d48da6d829fa3827e52 not found: ID does not exist"
Oct 30 22:33:49.805417 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:33:49.805387    4412 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw1-7476d02544/isvc-triton-predictor-795d5b7fd4-945xn"]
Oct 30 22:33:49.855795 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:33:49.855768    4412 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-gw1-7476d02544/isvc-triton-predictor-795d5b7fd4-945xn"]
Oct 30 22:33:51.625898 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:33:51.625866    4412 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="4cde3f35-4b9b-48c9-b03b-9888e545665b" path="/var/lib/kubelet/pods/4cde3f35-4b9b-48c9-b03b-9888e545665b/volumes"
Oct 30 22:33:53.474913 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:33:53.474882    4412 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-gw1-7476d02544/isvc-xgboost-predictor-655f7545bf-rz2bf"]
Oct 30 22:33:53.475294 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:33:53.475230    4412 cpu_manager.go:414] "containerMap: removing container" podUID="4cde3f35-4b9b-48c9-b03b-9888e545665b" containerName="kserve-container"
Oct 30 22:33:53.475294 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:33:53.475243    4412 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4cde3f35-4b9b-48c9-b03b-9888e545665b" containerName="kserve-container"
Oct 30 22:33:53.475294 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:33:53.475261    4412 cpu_manager.go:414] "containerMap: removing container" podUID="4cde3f35-4b9b-48c9-b03b-9888e545665b" containerName="storage-initializer"
Oct 30 22:33:53.475294 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:33:53.475266    4412 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4cde3f35-4b9b-48c9-b03b-9888e545665b" containerName="storage-initializer"
Oct 30 22:33:53.475294 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:33:53.475278    4412 cpu_manager.go:414] "containerMap: removing container" podUID="4cde3f35-4b9b-48c9-b03b-9888e545665b" containerName="kube-rbac-proxy"
Oct 30 22:33:53.475294 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:33:53.475283    4412 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4cde3f35-4b9b-48c9-b03b-9888e545665b" containerName="kube-rbac-proxy"
Oct 30 22:33:53.475480 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:33:53.475332    4412 memory_manager.go:365] "RemoveStaleState removing state" podUID="4cde3f35-4b9b-48c9-b03b-9888e545665b" containerName="kube-rbac-proxy"
Oct 30 22:33:53.475480 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:33:53.475342    4412 memory_manager.go:365] "RemoveStaleState removing state" podUID="4cde3f35-4b9b-48c9-b03b-9888e545665b" containerName="kserve-container"
Oct 30 22:33:53.505792 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:33:53.505772    4412 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-7476d02544/isvc-xgboost-predictor-655f7545bf-rz2bf"
Oct 30 22:33:53.549289 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:33:53.549258    4412 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/00bff02a-809a-44dd-966d-7f8fef0e1eb5-kube-rbac-proxy-sar-config\") pod \"isvc-xgboost-predictor-655f7545bf-rz2bf\" (UID: \"00bff02a-809a-44dd-966d-7f8fef0e1eb5\") " pod="e2e-gw1-7476d02544/isvc-xgboost-predictor-655f7545bf-rz2bf"
Oct 30 22:33:53.549411 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:33:53.549310    4412 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/00bff02a-809a-44dd-966d-7f8fef0e1eb5-proxy-tls\") pod \"isvc-xgboost-predictor-655f7545bf-rz2bf\" (UID: \"00bff02a-809a-44dd-966d-7f8fef0e1eb5\") " pod="e2e-gw1-7476d02544/isvc-xgboost-predictor-655f7545bf-rz2bf"
Oct 30 22:33:53.549465 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:33:53.549417    4412 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/00bff02a-809a-44dd-966d-7f8fef0e1eb5-kserve-provision-location\") pod \"isvc-xgboost-predictor-655f7545bf-rz2bf\" (UID: \"00bff02a-809a-44dd-966d-7f8fef0e1eb5\") " pod="e2e-gw1-7476d02544/isvc-xgboost-predictor-655f7545bf-rz2bf"
Oct 30 22:33:53.549526 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:33:53.549461    4412 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/00bff02a-809a-44dd-966d-7f8fef0e1eb5-cabundle-cert\") pod \"isvc-xgboost-predictor-655f7545bf-rz2bf\" (UID: \"00bff02a-809a-44dd-966d-7f8fef0e1eb5\") " pod="e2e-gw1-7476d02544/isvc-xgboost-predictor-655f7545bf-rz2bf"
Oct 30 22:33:53.549584 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:33:53.549554    4412 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zzn8k\" (UniqueName: \"kubernetes.io/projected/00bff02a-809a-44dd-966d-7f8fef0e1eb5-kube-api-access-zzn8k\") pod \"isvc-xgboost-predictor-655f7545bf-rz2bf\" (UID: \"00bff02a-809a-44dd-966d-7f8fef0e1eb5\") " pod="e2e-gw1-7476d02544/isvc-xgboost-predictor-655f7545bf-rz2bf"
Oct 30 22:33:53.559011 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:33:53.558995    4412 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw1-7476d02544\"/\"isvc-xgboost-kube-rbac-proxy-sar-config\""
Oct 30 22:33:53.559115 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:33:53.559098    4412 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw1-7476d02544\"/\"odh-kserve-custom-ca-bundle\""
Oct 30 22:33:53.559200 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:33:53.559102    4412 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw1-7476d02544\"/\"openshift-service-ca.crt\""
Oct 30 22:33:53.609527 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:33:53.609500    4412 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw1-7476d02544\"/\"seaweedfs-s3-creds\""
Oct 30 22:33:53.609527 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:33:53.609516    4412 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw1-7476d02544\"/\"kube-root-ca.crt\""
Oct 30 22:33:53.609659 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:33:53.609593    4412 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw1-7476d02544\"/\"isvc-xgboost-predictor-serving-cert\""
Oct 30 22:33:53.609697 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:33:53.609670    4412 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw1-7476d02544\"/\"default-dockercfg-cxfkn\""
Oct 30 22:33:53.610882 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:33:53.610864    4412 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw1-7476d02544/isvc-xgboost-predictor-655f7545bf-rz2bf"]
Oct 30 22:33:53.650602 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:33:53.650575    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/00bff02a-809a-44dd-966d-7f8fef0e1eb5-kube-rbac-proxy-sar-config\") pod \"isvc-xgboost-predictor-655f7545bf-rz2bf\" (UID: \"00bff02a-809a-44dd-966d-7f8fef0e1eb5\") " pod="e2e-gw1-7476d02544/isvc-xgboost-predictor-655f7545bf-rz2bf"
Oct 30 22:33:53.650742 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:33:53.650616    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/00bff02a-809a-44dd-966d-7f8fef0e1eb5-proxy-tls\") pod \"isvc-xgboost-predictor-655f7545bf-rz2bf\" (UID: \"00bff02a-809a-44dd-966d-7f8fef0e1eb5\") " pod="e2e-gw1-7476d02544/isvc-xgboost-predictor-655f7545bf-rz2bf"
Oct 30 22:33:53.650742 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:33:53.650668    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/00bff02a-809a-44dd-966d-7f8fef0e1eb5-kserve-provision-location\") pod \"isvc-xgboost-predictor-655f7545bf-rz2bf\" (UID: \"00bff02a-809a-44dd-966d-7f8fef0e1eb5\") " pod="e2e-gw1-7476d02544/isvc-xgboost-predictor-655f7545bf-rz2bf"
Oct 30 22:33:53.650742 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:33:53.650713    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/00bff02a-809a-44dd-966d-7f8fef0e1eb5-cabundle-cert\") pod \"isvc-xgboost-predictor-655f7545bf-rz2bf\" (UID: \"00bff02a-809a-44dd-966d-7f8fef0e1eb5\") " pod="e2e-gw1-7476d02544/isvc-xgboost-predictor-655f7545bf-rz2bf"
Oct 30 22:33:53.650742 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:33:53.650738    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zzn8k\" (UniqueName: \"kubernetes.io/projected/00bff02a-809a-44dd-966d-7f8fef0e1eb5-kube-api-access-zzn8k\") pod \"isvc-xgboost-predictor-655f7545bf-rz2bf\" (UID: \"00bff02a-809a-44dd-966d-7f8fef0e1eb5\") " pod="e2e-gw1-7476d02544/isvc-xgboost-predictor-655f7545bf-rz2bf"
Oct 30 22:33:53.651234 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:33:53.651202    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/00bff02a-809a-44dd-966d-7f8fef0e1eb5-kserve-provision-location\") pod \"isvc-xgboost-predictor-655f7545bf-rz2bf\" (UID: \"00bff02a-809a-44dd-966d-7f8fef0e1eb5\") " pod="e2e-gw1-7476d02544/isvc-xgboost-predictor-655f7545bf-rz2bf"
Oct 30 22:33:53.651544 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:33:53.651481    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/00bff02a-809a-44dd-966d-7f8fef0e1eb5-cabundle-cert\") pod \"isvc-xgboost-predictor-655f7545bf-rz2bf\" (UID: \"00bff02a-809a-44dd-966d-7f8fef0e1eb5\") " pod="e2e-gw1-7476d02544/isvc-xgboost-predictor-655f7545bf-rz2bf"
Oct 30 22:33:53.651544 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:33:53.651517    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/00bff02a-809a-44dd-966d-7f8fef0e1eb5-kube-rbac-proxy-sar-config\") pod \"isvc-xgboost-predictor-655f7545bf-rz2bf\" (UID: \"00bff02a-809a-44dd-966d-7f8fef0e1eb5\") " pod="e2e-gw1-7476d02544/isvc-xgboost-predictor-655f7545bf-rz2bf"
Oct 30 22:33:53.653472 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:33:53.653448    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/00bff02a-809a-44dd-966d-7f8fef0e1eb5-proxy-tls\") pod \"isvc-xgboost-predictor-655f7545bf-rz2bf\" (UID: \"00bff02a-809a-44dd-966d-7f8fef0e1eb5\") " pod="e2e-gw1-7476d02544/isvc-xgboost-predictor-655f7545bf-rz2bf"
Oct 30 22:33:53.708836 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:33:53.708814    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zzn8k\" (UniqueName: \"kubernetes.io/projected/00bff02a-809a-44dd-966d-7f8fef0e1eb5-kube-api-access-zzn8k\") pod \"isvc-xgboost-predictor-655f7545bf-rz2bf\" (UID: \"00bff02a-809a-44dd-966d-7f8fef0e1eb5\") " pod="e2e-gw1-7476d02544/isvc-xgboost-predictor-655f7545bf-rz2bf"
Oct 30 22:33:53.857097 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:33:53.857025    4412 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-7476d02544/isvc-xgboost-predictor-655f7545bf-rz2bf"
Oct 30 22:33:54.298676 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:33:54.298505    4412 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw1-7476d02544/isvc-xgboost-predictor-655f7545bf-rz2bf"]
Oct 30 22:33:54.301314 ip-10-0-131-202 kubenswrapper[4412]: W1030 22:33:54.301288    4412 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod00bff02a_809a_44dd_966d_7f8fef0e1eb5.slice/crio-eed80838c349494ba275c034b0bc71f9a0315498023e6751d97ca99d9b561a7f WatchSource:0}: Error finding container eed80838c349494ba275c034b0bc71f9a0315498023e6751d97ca99d9b561a7f: Status 404 returned error can't find the container with id eed80838c349494ba275c034b0bc71f9a0315498023e6751d97ca99d9b561a7f
Oct 30 22:33:54.660705 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:33:54.660619    4412 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-7476d02544/isvc-xgboost-predictor-655f7545bf-rz2bf" event={"ID":"00bff02a-809a-44dd-966d-7f8fef0e1eb5","Type":"ContainerStarted","Data":"f8282d5c835211cf9269e95803a348f965c58477933d67a5e90107b30b70e786"}
Oct 30 22:33:54.660705 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:33:54.660689    4412 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-7476d02544/isvc-xgboost-predictor-655f7545bf-rz2bf" event={"ID":"00bff02a-809a-44dd-966d-7f8fef0e1eb5","Type":"ContainerStarted","Data":"eed80838c349494ba275c034b0bc71f9a0315498023e6751d97ca99d9b561a7f"}
Oct 30 22:33:59.679603 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:33:59.679571    4412 generic.go:356] "Generic (PLEG): container finished" podID="00bff02a-809a-44dd-966d-7f8fef0e1eb5" containerID="f8282d5c835211cf9269e95803a348f965c58477933d67a5e90107b30b70e786" exitCode=0
Oct 30 22:33:59.680081 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:33:59.679644    4412 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-7476d02544/isvc-xgboost-predictor-655f7545bf-rz2bf" event={"ID":"00bff02a-809a-44dd-966d-7f8fef0e1eb5","Type":"ContainerDied","Data":"f8282d5c835211cf9269e95803a348f965c58477933d67a5e90107b30b70e786"}
Oct 30 22:34:04.451083 ip-10-0-131-202 kubenswrapper[4412]: E1030 22:34:04.451046    4412 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bkhmz" containerName="extractor"
Oct 30 22:34:26.781282 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:34:26.781246    4412 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-7476d02544/isvc-xgboost-predictor-655f7545bf-rz2bf" event={"ID":"00bff02a-809a-44dd-966d-7f8fef0e1eb5","Type":"ContainerStarted","Data":"5b201c378cdbad41f96f61f06bc3ad629080945fee98d1b482ccd49c6f51c849"}
Oct 30 22:34:26.781675 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:34:26.781308    4412 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-7476d02544/isvc-xgboost-predictor-655f7545bf-rz2bf" event={"ID":"00bff02a-809a-44dd-966d-7f8fef0e1eb5","Type":"ContainerStarted","Data":"770898c6cb04328bf0a0c02e4e6d7ebed83d6c09fcc8dc2fd827f0065de49fc3"}
Oct 30 22:34:26.781675 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:34:26.781490    4412 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw1-7476d02544/isvc-xgboost-predictor-655f7545bf-rz2bf"
Oct 30 22:34:26.952741 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:34:26.952670    4412 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-gw1-7476d02544/isvc-xgboost-predictor-655f7545bf-rz2bf" podStartSLOduration=3.299105015 podStartE2EDuration="33.952656234s" totalImagesPullingTime="26.187697346s" totalInitContainerRuntime="4.465853873s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:33:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:34:26.951007334 +0000 UTC m=+2487.455679299" watchObservedRunningTime="2026-10-30 22:34:26.952656234 +0000 UTC m=+2487.457328297"
Oct 30 22:34:27.785191 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:34:27.785156    4412 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw1-7476d02544/isvc-xgboost-predictor-655f7545bf-rz2bf"
Oct 30 22:34:27.786463 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:34:27.786435    4412 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-7476d02544/isvc-xgboost-predictor-655f7545bf-rz2bf" podUID="00bff02a-809a-44dd-966d-7f8fef0e1eb5" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.27:8080: connect: connection refused"
Oct 30 22:34:28.789098 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:34:28.789064    4412 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-7476d02544/isvc-xgboost-predictor-655f7545bf-rz2bf" podUID="00bff02a-809a-44dd-966d-7f8fef0e1eb5" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.27:8080: connect: connection refused"
Oct 30 22:34:33.793562 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:34:33.793534    4412 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw1-7476d02544/isvc-xgboost-predictor-655f7545bf-rz2bf"
Oct 30 22:34:33.793996 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:34:33.793973    4412 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-7476d02544/isvc-xgboost-predictor-655f7545bf-rz2bf" podUID="00bff02a-809a-44dd-966d-7f8fef0e1eb5" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.27:8080: connect: connection refused"
Oct 30 22:34:34.449140 ip-10-0-131-202 kubenswrapper[4412]: E1030 22:34:34.449074    4412 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bkhmz" containerName="extractor"
Oct 30 22:34:43.794779 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:34:43.794738    4412 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-7476d02544/isvc-xgboost-predictor-655f7545bf-rz2bf" podUID="00bff02a-809a-44dd-966d-7f8fef0e1eb5" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.27:8080: connect: connection refused"
Oct 30 22:34:53.794676 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:34:53.794640    4412 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-7476d02544/isvc-xgboost-predictor-655f7545bf-rz2bf" podUID="00bff02a-809a-44dd-966d-7f8fef0e1eb5" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.27:8080: connect: connection refused"
Oct 30 22:35:03.794023 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:35:03.793981    4412 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-7476d02544/isvc-xgboost-predictor-655f7545bf-rz2bf" podUID="00bff02a-809a-44dd-966d-7f8fef0e1eb5" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.27:8080: connect: connection refused"
Oct 30 22:35:04.448615 ip-10-0-131-202 kubenswrapper[4412]: E1030 22:35:04.448578    4412 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bkhmz" containerName="extractor"
Oct 30 22:35:13.794320 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:35:13.794283    4412 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-7476d02544/isvc-xgboost-predictor-655f7545bf-rz2bf" podUID="00bff02a-809a-44dd-966d-7f8fef0e1eb5" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.27:8080: connect: connection refused"
Oct 30 22:35:23.794352 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:35:23.794317    4412 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-7476d02544/isvc-xgboost-predictor-655f7545bf-rz2bf" podUID="00bff02a-809a-44dd-966d-7f8fef0e1eb5" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.27:8080: connect: connection refused"
Oct 30 22:35:33.794876 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:35:33.794843    4412 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw1-7476d02544/isvc-xgboost-predictor-655f7545bf-rz2bf"
Oct 30 22:35:34.447815 ip-10-0-131-202 kubenswrapper[4412]: E1030 22:35:34.447779    4412 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bkhmz" containerName="extractor"
Oct 30 22:35:44.220858 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:35:44.220784    4412 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw1-7476d02544/isvc-xgboost-predictor-655f7545bf-rz2bf"]
Oct 30 22:35:44.221253 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:35:44.221017    4412 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw1-7476d02544/isvc-xgboost-predictor-655f7545bf-rz2bf" podUID="00bff02a-809a-44dd-966d-7f8fef0e1eb5" containerName="kserve-container" containerID="cri-o://770898c6cb04328bf0a0c02e4e6d7ebed83d6c09fcc8dc2fd827f0065de49fc3" gracePeriod=30
Oct 30 22:35:44.221253 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:35:44.221068    4412 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw1-7476d02544/isvc-xgboost-predictor-655f7545bf-rz2bf" podUID="00bff02a-809a-44dd-966d-7f8fef0e1eb5" containerName="kube-rbac-proxy" containerID="cri-o://5b201c378cdbad41f96f61f06bc3ad629080945fee98d1b482ccd49c6f51c849" gracePeriod=30
Oct 30 22:35:45.066849 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:35:45.066812    4412 generic.go:356] "Generic (PLEG): container finished" podID="00bff02a-809a-44dd-966d-7f8fef0e1eb5" containerID="5b201c378cdbad41f96f61f06bc3ad629080945fee98d1b482ccd49c6f51c849" exitCode=0
Oct 30 22:35:45.067031 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:35:45.066889    4412 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-7476d02544/isvc-xgboost-predictor-655f7545bf-rz2bf" event={"ID":"00bff02a-809a-44dd-966d-7f8fef0e1eb5","Type":"ContainerDied","Data":"5b201c378cdbad41f96f61f06bc3ad629080945fee98d1b482ccd49c6f51c849"}
Oct 30 22:35:48.079615 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:35:48.079589    4412 generic.go:356] "Generic (PLEG): container finished" podID="00bff02a-809a-44dd-966d-7f8fef0e1eb5" containerID="770898c6cb04328bf0a0c02e4e6d7ebed83d6c09fcc8dc2fd827f0065de49fc3" exitCode=0
Oct 30 22:35:48.080007 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:35:48.079639    4412 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-7476d02544/isvc-xgboost-predictor-655f7545bf-rz2bf" event={"ID":"00bff02a-809a-44dd-966d-7f8fef0e1eb5","Type":"ContainerDied","Data":"770898c6cb04328bf0a0c02e4e6d7ebed83d6c09fcc8dc2fd827f0065de49fc3"}
Oct 30 22:35:48.080007 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:35:48.079699    4412 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-7476d02544/isvc-xgboost-predictor-655f7545bf-rz2bf" event={"ID":"00bff02a-809a-44dd-966d-7f8fef0e1eb5","Type":"ContainerDied","Data":"eed80838c349494ba275c034b0bc71f9a0315498023e6751d97ca99d9b561a7f"}
Oct 30 22:35:48.080007 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:35:48.079731    4412 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="eed80838c349494ba275c034b0bc71f9a0315498023e6751d97ca99d9b561a7f"
Oct 30 22:35:48.091298 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:35:48.091251    4412 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-7476d02544/isvc-xgboost-predictor-655f7545bf-rz2bf"
Oct 30 22:35:48.173931 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:35:48.173911    4412 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/00bff02a-809a-44dd-966d-7f8fef0e1eb5-kube-api-access-zzn8k\" (UniqueName: \"kubernetes.io/projected/00bff02a-809a-44dd-966d-7f8fef0e1eb5-kube-api-access-zzn8k\") pod \"00bff02a-809a-44dd-966d-7f8fef0e1eb5\" (UID: \"00bff02a-809a-44dd-966d-7f8fef0e1eb5\") "
Oct 30 22:35:48.174051 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:35:48.173954    4412 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/00bff02a-809a-44dd-966d-7f8fef0e1eb5-proxy-tls\" (UniqueName: \"kubernetes.io/secret/00bff02a-809a-44dd-966d-7f8fef0e1eb5-proxy-tls\") pod \"00bff02a-809a-44dd-966d-7f8fef0e1eb5\" (UID: \"00bff02a-809a-44dd-966d-7f8fef0e1eb5\") "
Oct 30 22:35:48.174051 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:35:48.173976    4412 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/00bff02a-809a-44dd-966d-7f8fef0e1eb5-kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/00bff02a-809a-44dd-966d-7f8fef0e1eb5-kube-rbac-proxy-sar-config\") pod \"00bff02a-809a-44dd-966d-7f8fef0e1eb5\" (UID: \"00bff02a-809a-44dd-966d-7f8fef0e1eb5\") "
Oct 30 22:35:48.174051 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:35:48.174014    4412 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/00bff02a-809a-44dd-966d-7f8fef0e1eb5-kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/00bff02a-809a-44dd-966d-7f8fef0e1eb5-kserve-provision-location\") pod \"00bff02a-809a-44dd-966d-7f8fef0e1eb5\" (UID: \"00bff02a-809a-44dd-966d-7f8fef0e1eb5\") "
Oct 30 22:35:48.174051 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:35:48.174040    4412 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/00bff02a-809a-44dd-966d-7f8fef0e1eb5-cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/00bff02a-809a-44dd-966d-7f8fef0e1eb5-cabundle-cert\") pod \"00bff02a-809a-44dd-966d-7f8fef0e1eb5\" (UID: \"00bff02a-809a-44dd-966d-7f8fef0e1eb5\") "
Oct 30 22:35:48.174401 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:35:48.174378    4412 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/00bff02a-809a-44dd-966d-7f8fef0e1eb5-kube-rbac-proxy-sar-config" pod "00bff02a-809a-44dd-966d-7f8fef0e1eb5" (UID: "00bff02a-809a-44dd-966d-7f8fef0e1eb5"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:35:48.174472 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:35:48.174389    4412 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/00bff02a-809a-44dd-966d-7f8fef0e1eb5-kserve-provision-location" pod "00bff02a-809a-44dd-966d-7f8fef0e1eb5" (UID: "00bff02a-809a-44dd-966d-7f8fef0e1eb5"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:35:48.174567 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:35:48.174538    4412 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/00bff02a-809a-44dd-966d-7f8fef0e1eb5-cabundle-cert" pod "00bff02a-809a-44dd-966d-7f8fef0e1eb5" (UID: "00bff02a-809a-44dd-966d-7f8fef0e1eb5"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:35:48.176172 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:35:48.176151    4412 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/00bff02a-809a-44dd-966d-7f8fef0e1eb5-proxy-tls" pod "00bff02a-809a-44dd-966d-7f8fef0e1eb5" (UID: "00bff02a-809a-44dd-966d-7f8fef0e1eb5"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:35:48.176261 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:35:48.176244    4412 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/00bff02a-809a-44dd-966d-7f8fef0e1eb5-kube-api-access-zzn8k" pod "00bff02a-809a-44dd-966d-7f8fef0e1eb5" (UID: "00bff02a-809a-44dd-966d-7f8fef0e1eb5"). InnerVolumeSpecName "kube-api-access-zzn8k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:35:48.275307 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:35:48.275280    4412 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zzn8k\" (UniqueName: \"kubernetes.io/projected/00bff02a-809a-44dd-966d-7f8fef0e1eb5-kube-api-access-zzn8k\") on node \"ip-10-0-131-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:35:48.275307 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:35:48.275302    4412 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/00bff02a-809a-44dd-966d-7f8fef0e1eb5-proxy-tls\") on node \"ip-10-0-131-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:35:48.275307 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:35:48.275312    4412 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/00bff02a-809a-44dd-966d-7f8fef0e1eb5-kube-rbac-proxy-sar-config\") on node \"ip-10-0-131-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:35:48.275498 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:35:48.275321    4412 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/00bff02a-809a-44dd-966d-7f8fef0e1eb5-kserve-provision-location\") on node \"ip-10-0-131-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:35:48.275498 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:35:48.275330    4412 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/00bff02a-809a-44dd-966d-7f8fef0e1eb5-cabundle-cert\") on node \"ip-10-0-131-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:35:49.083258 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:35:49.083229    4412 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-7476d02544/isvc-xgboost-predictor-655f7545bf-rz2bf"
Oct 30 22:35:49.299866 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:35:49.299829    4412 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw1-7476d02544/isvc-xgboost-predictor-655f7545bf-rz2bf"]
Oct 30 22:35:49.350353 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:35:49.350279    4412 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-gw1-7476d02544/isvc-xgboost-predictor-655f7545bf-rz2bf"]
Oct 30 22:35:49.625867 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:35:49.625784    4412 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="00bff02a-809a-44dd-966d-7f8fef0e1eb5" path="/var/lib/kubelet/pods/00bff02a-809a-44dd-966d-7f8fef0e1eb5/volumes"
Oct 30 22:36:04.448320 ip-10-0-131-202 kubenswrapper[4412]: E1030 22:36:04.448286    4412 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bkhmz" containerName="extractor"
Oct 30 22:36:34.448241 ip-10-0-131-202 kubenswrapper[4412]: E1030 22:36:34.448202    4412 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bkhmz" containerName="extractor"
Oct 30 22:37:04.447545 ip-10-0-131-202 kubenswrapper[4412]: E1030 22:37:04.447507    4412 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bkhmz" containerName="extractor"
Oct 30 22:37:13.971979 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:37:13.971901    4412 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-gw0-37d062d1dd/isvc-canary-create-predictor-86d48b7dcf-757cs"]
Oct 30 22:37:13.972502 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:37:13.972219    4412 cpu_manager.go:414] "containerMap: removing container" podUID="00bff02a-809a-44dd-966d-7f8fef0e1eb5" containerName="kube-rbac-proxy"
Oct 30 22:37:13.972502 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:37:13.972231    4412 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="00bff02a-809a-44dd-966d-7f8fef0e1eb5" containerName="kube-rbac-proxy"
Oct 30 22:37:13.972502 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:37:13.972251    4412 cpu_manager.go:414] "containerMap: removing container" podUID="00bff02a-809a-44dd-966d-7f8fef0e1eb5" containerName="kserve-container"
Oct 30 22:37:13.972502 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:37:13.972260    4412 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="00bff02a-809a-44dd-966d-7f8fef0e1eb5" containerName="kserve-container"
Oct 30 22:37:13.972502 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:37:13.972273    4412 cpu_manager.go:414] "containerMap: removing container" podUID="00bff02a-809a-44dd-966d-7f8fef0e1eb5" containerName="storage-initializer"
Oct 30 22:37:13.972502 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:37:13.972278    4412 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="00bff02a-809a-44dd-966d-7f8fef0e1eb5" containerName="storage-initializer"
Oct 30 22:37:13.972502 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:37:13.972334    4412 memory_manager.go:365] "RemoveStaleState removing state" podUID="00bff02a-809a-44dd-966d-7f8fef0e1eb5" containerName="kube-rbac-proxy"
Oct 30 22:37:13.972502 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:37:13.972347    4412 memory_manager.go:365] "RemoveStaleState removing state" podUID="00bff02a-809a-44dd-966d-7f8fef0e1eb5" containerName="kserve-container"
Oct 30 22:37:13.974171 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:37:13.974154    4412 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-37d062d1dd/isvc-canary-create-predictor-86d48b7dcf-757cs"
Oct 30 22:37:14.027934 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:37:14.027909    4412 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw0-37d062d1dd\"/\"odh-kserve-custom-ca-bundle\""
Oct 30 22:37:14.078419 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:37:14.078393    4412 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw0-37d062d1dd\"/\"openshift-service-ca.crt\""
Oct 30 22:37:14.078571 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:37:14.078357    4412 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw0-37d062d1dd\"/\"isvc-canary-create-kube-rbac-proxy-sar-config\""
Oct 30 22:37:14.078571 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:37:14.078497    4412 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw0-37d062d1dd\"/\"kube-root-ca.crt\""
Oct 30 22:37:14.078571 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:37:14.078517    4412 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw0-37d062d1dd\"/\"seaweedfs-s3-creds\""
Oct 30 22:37:14.078571 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:37:14.078527    4412 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw0-37d062d1dd\"/\"default-dockercfg-4v4s8\""
Oct 30 22:37:14.078571 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:37:14.078538    4412 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw0-37d062d1dd\"/\"isvc-canary-create-predictor-serving-cert\""
Oct 30 22:37:14.079930 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:37:14.079913    4412 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw0-37d062d1dd/isvc-canary-create-predictor-86d48b7dcf-757cs"]
Oct 30 22:37:14.134538 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:37:14.132546    4412 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8j65d\" (UniqueName: \"kubernetes.io/projected/bda39c14-aa41-4803-b809-03d58087972a-kube-api-access-8j65d\") pod \"isvc-canary-create-predictor-86d48b7dcf-757cs\" (UID: \"bda39c14-aa41-4803-b809-03d58087972a\") " pod="e2e-gw0-37d062d1dd/isvc-canary-create-predictor-86d48b7dcf-757cs"
Oct 30 22:37:14.134538 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:37:14.132621    4412 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/bda39c14-aa41-4803-b809-03d58087972a-kube-rbac-proxy-sar-config\") pod \"isvc-canary-create-predictor-86d48b7dcf-757cs\" (UID: \"bda39c14-aa41-4803-b809-03d58087972a\") " pod="e2e-gw0-37d062d1dd/isvc-canary-create-predictor-86d48b7dcf-757cs"
Oct 30 22:37:14.134538 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:37:14.132654    4412 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/bda39c14-aa41-4803-b809-03d58087972a-kserve-provision-location\") pod \"isvc-canary-create-predictor-86d48b7dcf-757cs\" (UID: \"bda39c14-aa41-4803-b809-03d58087972a\") " pod="e2e-gw0-37d062d1dd/isvc-canary-create-predictor-86d48b7dcf-757cs"
Oct 30 22:37:14.134538 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:37:14.132709    4412 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/bda39c14-aa41-4803-b809-03d58087972a-proxy-tls\") pod \"isvc-canary-create-predictor-86d48b7dcf-757cs\" (UID: \"bda39c14-aa41-4803-b809-03d58087972a\") " pod="e2e-gw0-37d062d1dd/isvc-canary-create-predictor-86d48b7dcf-757cs"
Oct 30 22:37:14.134538 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:37:14.132733    4412 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/bda39c14-aa41-4803-b809-03d58087972a-cabundle-cert\") pod \"isvc-canary-create-predictor-86d48b7dcf-757cs\" (UID: \"bda39c14-aa41-4803-b809-03d58087972a\") " pod="e2e-gw0-37d062d1dd/isvc-canary-create-predictor-86d48b7dcf-757cs"
Oct 30 22:37:14.233482 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:37:14.233408    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8j65d\" (UniqueName: \"kubernetes.io/projected/bda39c14-aa41-4803-b809-03d58087972a-kube-api-access-8j65d\") pod \"isvc-canary-create-predictor-86d48b7dcf-757cs\" (UID: \"bda39c14-aa41-4803-b809-03d58087972a\") " pod="e2e-gw0-37d062d1dd/isvc-canary-create-predictor-86d48b7dcf-757cs"
Oct 30 22:37:14.233482 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:37:14.233450    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/bda39c14-aa41-4803-b809-03d58087972a-kube-rbac-proxy-sar-config\") pod \"isvc-canary-create-predictor-86d48b7dcf-757cs\" (UID: \"bda39c14-aa41-4803-b809-03d58087972a\") " pod="e2e-gw0-37d062d1dd/isvc-canary-create-predictor-86d48b7dcf-757cs"
Oct 30 22:37:14.233482 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:37:14.233467    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/bda39c14-aa41-4803-b809-03d58087972a-kserve-provision-location\") pod \"isvc-canary-create-predictor-86d48b7dcf-757cs\" (UID: \"bda39c14-aa41-4803-b809-03d58087972a\") " pod="e2e-gw0-37d062d1dd/isvc-canary-create-predictor-86d48b7dcf-757cs"
Oct 30 22:37:14.233744 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:37:14.233496    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/bda39c14-aa41-4803-b809-03d58087972a-proxy-tls\") pod \"isvc-canary-create-predictor-86d48b7dcf-757cs\" (UID: \"bda39c14-aa41-4803-b809-03d58087972a\") " pod="e2e-gw0-37d062d1dd/isvc-canary-create-predictor-86d48b7dcf-757cs"
Oct 30 22:37:14.233744 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:37:14.233513    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/bda39c14-aa41-4803-b809-03d58087972a-cabundle-cert\") pod \"isvc-canary-create-predictor-86d48b7dcf-757cs\" (UID: \"bda39c14-aa41-4803-b809-03d58087972a\") " pod="e2e-gw0-37d062d1dd/isvc-canary-create-predictor-86d48b7dcf-757cs"
Oct 30 22:37:14.233883 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:37:14.233865    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/bda39c14-aa41-4803-b809-03d58087972a-kserve-provision-location\") pod \"isvc-canary-create-predictor-86d48b7dcf-757cs\" (UID: \"bda39c14-aa41-4803-b809-03d58087972a\") " pod="e2e-gw0-37d062d1dd/isvc-canary-create-predictor-86d48b7dcf-757cs"
Oct 30 22:37:14.234178 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:37:14.234150    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/bda39c14-aa41-4803-b809-03d58087972a-cabundle-cert\") pod \"isvc-canary-create-predictor-86d48b7dcf-757cs\" (UID: \"bda39c14-aa41-4803-b809-03d58087972a\") " pod="e2e-gw0-37d062d1dd/isvc-canary-create-predictor-86d48b7dcf-757cs"
Oct 30 22:37:14.234302 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:37:14.234242    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/bda39c14-aa41-4803-b809-03d58087972a-kube-rbac-proxy-sar-config\") pod \"isvc-canary-create-predictor-86d48b7dcf-757cs\" (UID: \"bda39c14-aa41-4803-b809-03d58087972a\") " pod="e2e-gw0-37d062d1dd/isvc-canary-create-predictor-86d48b7dcf-757cs"
Oct 30 22:37:14.236188 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:37:14.236171    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/bda39c14-aa41-4803-b809-03d58087972a-proxy-tls\") pod \"isvc-canary-create-predictor-86d48b7dcf-757cs\" (UID: \"bda39c14-aa41-4803-b809-03d58087972a\") " pod="e2e-gw0-37d062d1dd/isvc-canary-create-predictor-86d48b7dcf-757cs"
Oct 30 22:37:14.291950 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:37:14.291927    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8j65d\" (UniqueName: \"kubernetes.io/projected/bda39c14-aa41-4803-b809-03d58087972a-kube-api-access-8j65d\") pod \"isvc-canary-create-predictor-86d48b7dcf-757cs\" (UID: \"bda39c14-aa41-4803-b809-03d58087972a\") " pod="e2e-gw0-37d062d1dd/isvc-canary-create-predictor-86d48b7dcf-757cs"
Oct 30 22:37:14.586862 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:37:14.586763    4412 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-37d062d1dd/isvc-canary-create-predictor-86d48b7dcf-757cs"
Oct 30 22:37:15.008083 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:37:15.008058    4412 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw0-37d062d1dd/isvc-canary-create-predictor-86d48b7dcf-757cs"]
Oct 30 22:37:15.010189 ip-10-0-131-202 kubenswrapper[4412]: W1030 22:37:15.010158    4412 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbda39c14_aa41_4803_b809_03d58087972a.slice/crio-7b8adf6eb02e1bbcabd4e59e7b00d0834f1967997db8a86cfe0428692cc0b0ae WatchSource:0}: Error finding container 7b8adf6eb02e1bbcabd4e59e7b00d0834f1967997db8a86cfe0428692cc0b0ae: Status 404 returned error can't find the container with id 7b8adf6eb02e1bbcabd4e59e7b00d0834f1967997db8a86cfe0428692cc0b0ae
Oct 30 22:37:15.392211 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:37:15.392113    4412 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-37d062d1dd/isvc-canary-create-predictor-86d48b7dcf-757cs" event={"ID":"bda39c14-aa41-4803-b809-03d58087972a","Type":"ContainerStarted","Data":"7ba026a3a2f00e8c9c0858c83342bb29d51d369b2790baf77328b5f51397926e"}
Oct 30 22:37:15.392211 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:37:15.392191    4412 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-37d062d1dd/isvc-canary-create-predictor-86d48b7dcf-757cs" event={"ID":"bda39c14-aa41-4803-b809-03d58087972a","Type":"ContainerStarted","Data":"7b8adf6eb02e1bbcabd4e59e7b00d0834f1967997db8a86cfe0428692cc0b0ae"}
Oct 30 22:37:19.408171 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:37:19.408071    4412 generic.go:356] "Generic (PLEG): container finished" podID="bda39c14-aa41-4803-b809-03d58087972a" containerID="7ba026a3a2f00e8c9c0858c83342bb29d51d369b2790baf77328b5f51397926e" exitCode=0
Oct 30 22:37:19.408171 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:37:19.408150    4412 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-37d062d1dd/isvc-canary-create-predictor-86d48b7dcf-757cs" event={"ID":"bda39c14-aa41-4803-b809-03d58087972a","Type":"ContainerDied","Data":"7ba026a3a2f00e8c9c0858c83342bb29d51d369b2790baf77328b5f51397926e"}
Oct 30 22:37:19.409247 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:37:19.409231    4412 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 22:37:34.451805 ip-10-0-131-202 kubenswrapper[4412]: E1030 22:37:34.451770    4412 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bkhmz" containerName="extractor"
Oct 30 22:37:34.479143 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:37:34.479097    4412 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-37d062d1dd/isvc-canary-create-predictor-86d48b7dcf-757cs" event={"ID":"bda39c14-aa41-4803-b809-03d58087972a","Type":"ContainerStarted","Data":"895da4203d66b98077ac28875337cd6cfa3190539872824568865bc064417a59"}
Oct 30 22:37:34.479278 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:37:34.479175    4412 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-37d062d1dd/isvc-canary-create-predictor-86d48b7dcf-757cs" event={"ID":"bda39c14-aa41-4803-b809-03d58087972a","Type":"ContainerStarted","Data":"b4d82268319f2d4109cdf16095ed3d3220f46e60af1b08124d4359e635af28b6"}
Oct 30 22:37:34.479394 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:37:34.479375    4412 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw0-37d062d1dd/isvc-canary-create-predictor-86d48b7dcf-757cs"
Oct 30 22:37:34.601189 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:37:34.601110    4412 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-gw0-37d062d1dd/isvc-canary-create-predictor-86d48b7dcf-757cs" podStartSLOduration=2.901629348 podStartE2EDuration="21.601097047s" totalImagesPullingTime="14.799954603s" totalInitContainerRuntime="3.899513096s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:37:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:37:34.600097893 +0000 UTC m=+2675.104769844" watchObservedRunningTime="2026-10-30 22:37:34.601097047 +0000 UTC m=+2675.105768999"
Oct 30 22:37:35.482965 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:37:35.482932    4412 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw0-37d062d1dd/isvc-canary-create-predictor-86d48b7dcf-757cs"
Oct 30 22:37:35.484399 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:37:35.484371    4412 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-37d062d1dd/isvc-canary-create-predictor-86d48b7dcf-757cs" podUID="bda39c14-aa41-4803-b809-03d58087972a" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.28:8080: connect: connection refused"
Oct 30 22:37:36.486168 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:37:36.486113    4412 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-37d062d1dd/isvc-canary-create-predictor-86d48b7dcf-757cs" podUID="bda39c14-aa41-4803-b809-03d58087972a" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.28:8080: connect: connection refused"
Oct 30 22:37:41.490465 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:37:41.490433    4412 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw0-37d062d1dd/isvc-canary-create-predictor-86d48b7dcf-757cs"
Oct 30 22:37:41.491042 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:37:41.491015    4412 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-37d062d1dd/isvc-canary-create-predictor-86d48b7dcf-757cs" podUID="bda39c14-aa41-4803-b809-03d58087972a" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.28:8080: connect: connection refused"
Oct 30 22:37:51.491060 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:37:51.491023    4412 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-37d062d1dd/isvc-canary-create-predictor-86d48b7dcf-757cs" podUID="bda39c14-aa41-4803-b809-03d58087972a" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.28:8080: connect: connection refused"
Oct 30 22:38:01.491518 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:38:01.491475    4412 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-37d062d1dd/isvc-canary-create-predictor-86d48b7dcf-757cs" podUID="bda39c14-aa41-4803-b809-03d58087972a" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.28:8080: connect: connection refused"
Oct 30 22:38:04.448593 ip-10-0-131-202 kubenswrapper[4412]: E1030 22:38:04.448555    4412 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bkhmz" containerName="extractor"
Oct 30 22:38:11.491269 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:38:11.491231    4412 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-37d062d1dd/isvc-canary-create-predictor-86d48b7dcf-757cs" podUID="bda39c14-aa41-4803-b809-03d58087972a" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.28:8080: connect: connection refused"
Oct 30 22:38:21.491240 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:38:21.491196    4412 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-37d062d1dd/isvc-canary-create-predictor-86d48b7dcf-757cs" podUID="bda39c14-aa41-4803-b809-03d58087972a" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.28:8080: connect: connection refused"
Oct 30 22:38:31.491088 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:38:31.491050    4412 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-37d062d1dd/isvc-canary-create-predictor-86d48b7dcf-757cs" podUID="bda39c14-aa41-4803-b809-03d58087972a" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.28:8080: connect: connection refused"
Oct 30 22:38:34.448656 ip-10-0-131-202 kubenswrapper[4412]: E1030 22:38:34.448555    4412 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bkhmz" containerName="extractor"
Oct 30 22:38:41.491224 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:38:41.491145    4412 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-37d062d1dd/isvc-canary-create-predictor-86d48b7dcf-757cs" podUID="bda39c14-aa41-4803-b809-03d58087972a" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.28:8080: connect: connection refused"
Oct 30 22:38:51.492038 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:38:51.492009    4412 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw0-37d062d1dd/isvc-canary-create-predictor-86d48b7dcf-757cs"
Oct 30 22:38:52.301949 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:38:52.301907    4412 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw0-37d062d1dd/isvc-canary-create-predictor-86d48b7dcf-757cs"]
Oct 30 22:38:52.302205 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:38:52.302179    4412 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw0-37d062d1dd/isvc-canary-create-predictor-86d48b7dcf-757cs" podUID="bda39c14-aa41-4803-b809-03d58087972a" containerName="kserve-container" containerID="cri-o://b4d82268319f2d4109cdf16095ed3d3220f46e60af1b08124d4359e635af28b6" gracePeriod=30
Oct 30 22:38:52.302307 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:38:52.302249    4412 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw0-37d062d1dd/isvc-canary-create-predictor-86d48b7dcf-757cs" podUID="bda39c14-aa41-4803-b809-03d58087972a" containerName="kube-rbac-proxy" containerID="cri-o://895da4203d66b98077ac28875337cd6cfa3190539872824568865bc064417a59" gracePeriod=30
Oct 30 22:38:52.763869 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:38:52.763830    4412 generic.go:356] "Generic (PLEG): container finished" podID="bda39c14-aa41-4803-b809-03d58087972a" containerID="895da4203d66b98077ac28875337cd6cfa3190539872824568865bc064417a59" exitCode=0
Oct 30 22:38:52.764290 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:38:52.763897    4412 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-37d062d1dd/isvc-canary-create-predictor-86d48b7dcf-757cs" event={"ID":"bda39c14-aa41-4803-b809-03d58087972a","Type":"ContainerDied","Data":"895da4203d66b98077ac28875337cd6cfa3190539872824568865bc064417a59"}
Oct 30 22:38:56.486708 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:38:56.486667    4412 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-37d062d1dd/isvc-canary-create-predictor-86d48b7dcf-757cs" podUID="bda39c14-aa41-4803-b809-03d58087972a" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.28:8643/healthz\": dial tcp 10.133.0.28:8643: connect: connection refused"
Oct 30 22:38:56.780196 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:38:56.780166    4412 generic.go:356] "Generic (PLEG): container finished" podID="bda39c14-aa41-4803-b809-03d58087972a" containerID="b4d82268319f2d4109cdf16095ed3d3220f46e60af1b08124d4359e635af28b6" exitCode=0
Oct 30 22:38:56.780327 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:38:56.780237    4412 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-37d062d1dd/isvc-canary-create-predictor-86d48b7dcf-757cs" event={"ID":"bda39c14-aa41-4803-b809-03d58087972a","Type":"ContainerDied","Data":"b4d82268319f2d4109cdf16095ed3d3220f46e60af1b08124d4359e635af28b6"}
Oct 30 22:38:56.840157 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:38:56.840132    4412 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-37d062d1dd/isvc-canary-create-predictor-86d48b7dcf-757cs"
Oct 30 22:38:56.942662 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:38:56.942635    4412 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/bda39c14-aa41-4803-b809-03d58087972a-kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/bda39c14-aa41-4803-b809-03d58087972a-kube-rbac-proxy-sar-config\") pod \"bda39c14-aa41-4803-b809-03d58087972a\" (UID: \"bda39c14-aa41-4803-b809-03d58087972a\") "
Oct 30 22:38:56.942812 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:38:56.942679    4412 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/bda39c14-aa41-4803-b809-03d58087972a-proxy-tls\" (UniqueName: \"kubernetes.io/secret/bda39c14-aa41-4803-b809-03d58087972a-proxy-tls\") pod \"bda39c14-aa41-4803-b809-03d58087972a\" (UID: \"bda39c14-aa41-4803-b809-03d58087972a\") "
Oct 30 22:38:56.942812 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:38:56.942734    4412 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/bda39c14-aa41-4803-b809-03d58087972a-kube-api-access-8j65d\" (UniqueName: \"kubernetes.io/projected/bda39c14-aa41-4803-b809-03d58087972a-kube-api-access-8j65d\") pod \"bda39c14-aa41-4803-b809-03d58087972a\" (UID: \"bda39c14-aa41-4803-b809-03d58087972a\") "
Oct 30 22:38:56.942812 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:38:56.942793    4412 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bda39c14-aa41-4803-b809-03d58087972a-kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/bda39c14-aa41-4803-b809-03d58087972a-kserve-provision-location\") pod \"bda39c14-aa41-4803-b809-03d58087972a\" (UID: \"bda39c14-aa41-4803-b809-03d58087972a\") "
Oct 30 22:38:56.943014 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:38:56.942979    4412 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/bda39c14-aa41-4803-b809-03d58087972a-cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/bda39c14-aa41-4803-b809-03d58087972a-cabundle-cert\") pod \"bda39c14-aa41-4803-b809-03d58087972a\" (UID: \"bda39c14-aa41-4803-b809-03d58087972a\") "
Oct 30 22:38:56.943090 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:38:56.943033    4412 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bda39c14-aa41-4803-b809-03d58087972a-kube-rbac-proxy-sar-config" pod "bda39c14-aa41-4803-b809-03d58087972a" (UID: "bda39c14-aa41-4803-b809-03d58087972a"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:38:56.943173 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:38:56.943092    4412 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bda39c14-aa41-4803-b809-03d58087972a-kserve-provision-location" pod "bda39c14-aa41-4803-b809-03d58087972a" (UID: "bda39c14-aa41-4803-b809-03d58087972a"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:38:56.943268 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:38:56.943248    4412 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/bda39c14-aa41-4803-b809-03d58087972a-kserve-provision-location\") on node \"ip-10-0-131-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:38:56.943314 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:38:56.943276    4412 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/bda39c14-aa41-4803-b809-03d58087972a-kube-rbac-proxy-sar-config\") on node \"ip-10-0-131-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:38:56.943351 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:38:56.943335    4412 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bda39c14-aa41-4803-b809-03d58087972a-cabundle-cert" pod "bda39c14-aa41-4803-b809-03d58087972a" (UID: "bda39c14-aa41-4803-b809-03d58087972a"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:38:56.944910 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:38:56.944890    4412 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bda39c14-aa41-4803-b809-03d58087972a-kube-api-access-8j65d" pod "bda39c14-aa41-4803-b809-03d58087972a" (UID: "bda39c14-aa41-4803-b809-03d58087972a"). InnerVolumeSpecName "kube-api-access-8j65d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:38:56.945003 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:38:56.944985    4412 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bda39c14-aa41-4803-b809-03d58087972a-proxy-tls" pod "bda39c14-aa41-4803-b809-03d58087972a" (UID: "bda39c14-aa41-4803-b809-03d58087972a"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:38:57.043729 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:38:57.043677    4412 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/bda39c14-aa41-4803-b809-03d58087972a-cabundle-cert\") on node \"ip-10-0-131-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:38:57.043729 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:38:57.043696    4412 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/bda39c14-aa41-4803-b809-03d58087972a-proxy-tls\") on node \"ip-10-0-131-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:38:57.043729 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:38:57.043708    4412 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8j65d\" (UniqueName: \"kubernetes.io/projected/bda39c14-aa41-4803-b809-03d58087972a-kube-api-access-8j65d\") on node \"ip-10-0-131-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:38:57.785920 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:38:57.785875    4412 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-37d062d1dd/isvc-canary-create-predictor-86d48b7dcf-757cs" event={"ID":"bda39c14-aa41-4803-b809-03d58087972a","Type":"ContainerDied","Data":"7b8adf6eb02e1bbcabd4e59e7b00d0834f1967997db8a86cfe0428692cc0b0ae"}
Oct 30 22:38:57.785920 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:38:57.785907    4412 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-37d062d1dd/isvc-canary-create-predictor-86d48b7dcf-757cs"
Oct 30 22:38:57.786469 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:38:57.785938    4412 scope.go:122] "RemoveContainer" containerID="895da4203d66b98077ac28875337cd6cfa3190539872824568865bc064417a59"
Oct 30 22:38:57.793944 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:38:57.793919    4412 scope.go:122] "RemoveContainer" containerID="b4d82268319f2d4109cdf16095ed3d3220f46e60af1b08124d4359e635af28b6"
Oct 30 22:38:57.803183 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:38:57.803142    4412 scope.go:122] "RemoveContainer" containerID="7ba026a3a2f00e8c9c0858c83342bb29d51d369b2790baf77328b5f51397926e"
Oct 30 22:38:57.951664 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:38:57.951580    4412 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw0-37d062d1dd/isvc-canary-create-predictor-86d48b7dcf-757cs"]
Oct 30 22:38:58.001800 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:38:58.001772    4412 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-gw0-37d062d1dd/isvc-canary-create-predictor-86d48b7dcf-757cs"]
Oct 30 22:38:59.625905 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:38:59.625876    4412 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="bda39c14-aa41-4803-b809-03d58087972a" path="/var/lib/kubelet/pods/bda39c14-aa41-4803-b809-03d58087972a/volumes"
Oct 30 22:39:04.447564 ip-10-0-131-202 kubenswrapper[4412]: E1030 22:39:04.447525    4412 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bkhmz" containerName="extractor"
Oct 30 22:39:13.146824 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:39:13.146791    4412 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-gw0-37d062d1dd/isvc-canary-promote-predictor-58d794bb6f-8j72d"]
Oct 30 22:39:13.147233 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:39:13.147100    4412 cpu_manager.go:414] "containerMap: removing container" podUID="bda39c14-aa41-4803-b809-03d58087972a" containerName="kserve-container"
Oct 30 22:39:13.147233 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:39:13.147111    4412 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="bda39c14-aa41-4803-b809-03d58087972a" containerName="kserve-container"
Oct 30 22:39:13.147233 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:39:13.147145    4412 cpu_manager.go:414] "containerMap: removing container" podUID="bda39c14-aa41-4803-b809-03d58087972a" containerName="kube-rbac-proxy"
Oct 30 22:39:13.147233 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:39:13.147151    4412 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="bda39c14-aa41-4803-b809-03d58087972a" containerName="kube-rbac-proxy"
Oct 30 22:39:13.147233 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:39:13.147164    4412 cpu_manager.go:414] "containerMap: removing container" podUID="bda39c14-aa41-4803-b809-03d58087972a" containerName="storage-initializer"
Oct 30 22:39:13.147233 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:39:13.147169    4412 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="bda39c14-aa41-4803-b809-03d58087972a" containerName="storage-initializer"
Oct 30 22:39:13.147233 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:39:13.147225    4412 memory_manager.go:365] "RemoveStaleState removing state" podUID="bda39c14-aa41-4803-b809-03d58087972a" containerName="kserve-container"
Oct 30 22:39:13.147233 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:39:13.147233    4412 memory_manager.go:365] "RemoveStaleState removing state" podUID="bda39c14-aa41-4803-b809-03d58087972a" containerName="kube-rbac-proxy"
Oct 30 22:39:13.149878 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:39:13.149862    4412 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-37d062d1dd/isvc-canary-promote-predictor-58d794bb6f-8j72d"
Oct 30 22:39:13.203268 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:39:13.203245    4412 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw0-37d062d1dd\"/\"odh-kserve-custom-ca-bundle\""
Oct 30 22:39:13.203408 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:39:13.203356    4412 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw0-37d062d1dd\"/\"kube-root-ca.crt\""
Oct 30 22:39:13.203448 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:39:13.203437    4412 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw0-37d062d1dd\"/\"isvc-canary-promote-kube-rbac-proxy-sar-config\""
Oct 30 22:39:13.253969 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:39:13.253948    4412 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw0-37d062d1dd\"/\"openshift-service-ca.crt\""
Oct 30 22:39:13.254190 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:39:13.253986    4412 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw0-37d062d1dd\"/\"isvc-canary-promote-predictor-serving-cert\""
Oct 30 22:39:13.254190 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:39:13.254045    4412 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw0-37d062d1dd\"/\"default-dockercfg-4v4s8\""
Oct 30 22:39:13.254190 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:39:13.254061    4412 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw0-37d062d1dd\"/\"seaweedfs-s3-creds\""
Oct 30 22:39:13.255372 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:39:13.255350    4412 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw0-37d062d1dd/isvc-canary-promote-predictor-58d794bb6f-8j72d"]
Oct 30 22:39:13.258456 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:39:13.258440    4412 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6rhln\" (UniqueName: \"kubernetes.io/projected/cf8b5667-ca48-460f-84d5-e6ee6e61c068-kube-api-access-6rhln\") pod \"isvc-canary-promote-predictor-58d794bb6f-8j72d\" (UID: \"cf8b5667-ca48-460f-84d5-e6ee6e61c068\") " pod="e2e-gw0-37d062d1dd/isvc-canary-promote-predictor-58d794bb6f-8j72d"
Oct 30 22:39:13.258528 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:39:13.258468    4412 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/cf8b5667-ca48-460f-84d5-e6ee6e61c068-cabundle-cert\") pod \"isvc-canary-promote-predictor-58d794bb6f-8j72d\" (UID: \"cf8b5667-ca48-460f-84d5-e6ee6e61c068\") " pod="e2e-gw0-37d062d1dd/isvc-canary-promote-predictor-58d794bb6f-8j72d"
Oct 30 22:39:13.258528 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:39:13.258505    4412 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/cf8b5667-ca48-460f-84d5-e6ee6e61c068-proxy-tls\") pod \"isvc-canary-promote-predictor-58d794bb6f-8j72d\" (UID: \"cf8b5667-ca48-460f-84d5-e6ee6e61c068\") " pod="e2e-gw0-37d062d1dd/isvc-canary-promote-predictor-58d794bb6f-8j72d"
Oct 30 22:39:13.258528 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:39:13.258522    4412 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/cf8b5667-ca48-460f-84d5-e6ee6e61c068-kube-rbac-proxy-sar-config\") pod \"isvc-canary-promote-predictor-58d794bb6f-8j72d\" (UID: \"cf8b5667-ca48-460f-84d5-e6ee6e61c068\") " pod="e2e-gw0-37d062d1dd/isvc-canary-promote-predictor-58d794bb6f-8j72d"
Oct 30 22:39:13.258675 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:39:13.258652    4412 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/cf8b5667-ca48-460f-84d5-e6ee6e61c068-kserve-provision-location\") pod \"isvc-canary-promote-predictor-58d794bb6f-8j72d\" (UID: \"cf8b5667-ca48-460f-84d5-e6ee6e61c068\") " pod="e2e-gw0-37d062d1dd/isvc-canary-promote-predictor-58d794bb6f-8j72d"
Oct 30 22:39:13.359422 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:39:13.359392    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/cf8b5667-ca48-460f-84d5-e6ee6e61c068-cabundle-cert\") pod \"isvc-canary-promote-predictor-58d794bb6f-8j72d\" (UID: \"cf8b5667-ca48-460f-84d5-e6ee6e61c068\") " pod="e2e-gw0-37d062d1dd/isvc-canary-promote-predictor-58d794bb6f-8j72d"
Oct 30 22:39:13.359568 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:39:13.359451    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/cf8b5667-ca48-460f-84d5-e6ee6e61c068-proxy-tls\") pod \"isvc-canary-promote-predictor-58d794bb6f-8j72d\" (UID: \"cf8b5667-ca48-460f-84d5-e6ee6e61c068\") " pod="e2e-gw0-37d062d1dd/isvc-canary-promote-predictor-58d794bb6f-8j72d"
Oct 30 22:39:13.359568 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:39:13.359483    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/cf8b5667-ca48-460f-84d5-e6ee6e61c068-kube-rbac-proxy-sar-config\") pod \"isvc-canary-promote-predictor-58d794bb6f-8j72d\" (UID: \"cf8b5667-ca48-460f-84d5-e6ee6e61c068\") " pod="e2e-gw0-37d062d1dd/isvc-canary-promote-predictor-58d794bb6f-8j72d"
Oct 30 22:39:13.359568 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:39:13.359516    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/cf8b5667-ca48-460f-84d5-e6ee6e61c068-kserve-provision-location\") pod \"isvc-canary-promote-predictor-58d794bb6f-8j72d\" (UID: \"cf8b5667-ca48-460f-84d5-e6ee6e61c068\") " pod="e2e-gw0-37d062d1dd/isvc-canary-promote-predictor-58d794bb6f-8j72d"
Oct 30 22:39:13.359568 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:39:13.359565    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6rhln\" (UniqueName: \"kubernetes.io/projected/cf8b5667-ca48-460f-84d5-e6ee6e61c068-kube-api-access-6rhln\") pod \"isvc-canary-promote-predictor-58d794bb6f-8j72d\" (UID: \"cf8b5667-ca48-460f-84d5-e6ee6e61c068\") " pod="e2e-gw0-37d062d1dd/isvc-canary-promote-predictor-58d794bb6f-8j72d"
Oct 30 22:39:13.360022 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:39:13.359994    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/cf8b5667-ca48-460f-84d5-e6ee6e61c068-kserve-provision-location\") pod \"isvc-canary-promote-predictor-58d794bb6f-8j72d\" (UID: \"cf8b5667-ca48-460f-84d5-e6ee6e61c068\") " pod="e2e-gw0-37d062d1dd/isvc-canary-promote-predictor-58d794bb6f-8j72d"
Oct 30 22:39:13.360198 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:39:13.360180    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/cf8b5667-ca48-460f-84d5-e6ee6e61c068-cabundle-cert\") pod \"isvc-canary-promote-predictor-58d794bb6f-8j72d\" (UID: \"cf8b5667-ca48-460f-84d5-e6ee6e61c068\") " pod="e2e-gw0-37d062d1dd/isvc-canary-promote-predictor-58d794bb6f-8j72d"
Oct 30 22:39:13.360257 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:39:13.360237    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/cf8b5667-ca48-460f-84d5-e6ee6e61c068-kube-rbac-proxy-sar-config\") pod \"isvc-canary-promote-predictor-58d794bb6f-8j72d\" (UID: \"cf8b5667-ca48-460f-84d5-e6ee6e61c068\") " pod="e2e-gw0-37d062d1dd/isvc-canary-promote-predictor-58d794bb6f-8j72d"
Oct 30 22:39:13.362203 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:39:13.362187    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/cf8b5667-ca48-460f-84d5-e6ee6e61c068-proxy-tls\") pod \"isvc-canary-promote-predictor-58d794bb6f-8j72d\" (UID: \"cf8b5667-ca48-460f-84d5-e6ee6e61c068\") " pod="e2e-gw0-37d062d1dd/isvc-canary-promote-predictor-58d794bb6f-8j72d"
Oct 30 22:39:13.419169 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:39:13.419082    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6rhln\" (UniqueName: \"kubernetes.io/projected/cf8b5667-ca48-460f-84d5-e6ee6e61c068-kube-api-access-6rhln\") pod \"isvc-canary-promote-predictor-58d794bb6f-8j72d\" (UID: \"cf8b5667-ca48-460f-84d5-e6ee6e61c068\") " pod="e2e-gw0-37d062d1dd/isvc-canary-promote-predictor-58d794bb6f-8j72d"
Oct 30 22:39:13.462847 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:39:13.462814    4412 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-37d062d1dd/isvc-canary-promote-predictor-58d794bb6f-8j72d"
Oct 30 22:39:13.882029 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:39:13.882006    4412 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw0-37d062d1dd/isvc-canary-promote-predictor-58d794bb6f-8j72d"]
Oct 30 22:39:13.884073 ip-10-0-131-202 kubenswrapper[4412]: W1030 22:39:13.884045    4412 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcf8b5667_ca48_460f_84d5_e6ee6e61c068.slice/crio-6f4d8f7ccca49a991db06aab7a453af4d515cd9aacb6b90015a8f16a9c970330 WatchSource:0}: Error finding container 6f4d8f7ccca49a991db06aab7a453af4d515cd9aacb6b90015a8f16a9c970330: Status 404 returned error can't find the container with id 6f4d8f7ccca49a991db06aab7a453af4d515cd9aacb6b90015a8f16a9c970330
Oct 30 22:39:14.850297 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:39:14.850265    4412 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-37d062d1dd/isvc-canary-promote-predictor-58d794bb6f-8j72d" event={"ID":"cf8b5667-ca48-460f-84d5-e6ee6e61c068","Type":"ContainerStarted","Data":"2647831515c72c725f7248e0d324334530e283a28f2444530da0903788d66569"}
Oct 30 22:39:14.850722 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:39:14.850323    4412 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-37d062d1dd/isvc-canary-promote-predictor-58d794bb6f-8j72d" event={"ID":"cf8b5667-ca48-460f-84d5-e6ee6e61c068","Type":"ContainerStarted","Data":"6f4d8f7ccca49a991db06aab7a453af4d515cd9aacb6b90015a8f16a9c970330"}
Oct 30 22:39:17.862863 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:39:17.862783    4412 generic.go:356] "Generic (PLEG): container finished" podID="cf8b5667-ca48-460f-84d5-e6ee6e61c068" containerID="2647831515c72c725f7248e0d324334530e283a28f2444530da0903788d66569" exitCode=0
Oct 30 22:39:17.862863 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:39:17.862857    4412 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-37d062d1dd/isvc-canary-promote-predictor-58d794bb6f-8j72d" event={"ID":"cf8b5667-ca48-460f-84d5-e6ee6e61c068","Type":"ContainerDied","Data":"2647831515c72c725f7248e0d324334530e283a28f2444530da0903788d66569"}
Oct 30 22:39:18.868558 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:39:18.868519    4412 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-37d062d1dd/isvc-canary-promote-predictor-58d794bb6f-8j72d" event={"ID":"cf8b5667-ca48-460f-84d5-e6ee6e61c068","Type":"ContainerStarted","Data":"a6c1476e9055709d57dbed4e59f826c259bc0539de176c4769b86a2b1035526c"}
Oct 30 22:39:18.869069 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:39:18.868590    4412 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-37d062d1dd/isvc-canary-promote-predictor-58d794bb6f-8j72d" event={"ID":"cf8b5667-ca48-460f-84d5-e6ee6e61c068","Type":"ContainerStarted","Data":"10bed3e9713eb9c2c43455d2f07411f56abfe71df9a716defea3e175418e305d"}
Oct 30 22:39:18.869069 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:39:18.868806    4412 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw0-37d062d1dd/isvc-canary-promote-predictor-58d794bb6f-8j72d"
Oct 30 22:39:18.990011 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:39:18.989962    4412 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-gw0-37d062d1dd/isvc-canary-promote-predictor-58d794bb6f-8j72d" podStartSLOduration=2.319531058 podStartE2EDuration="5.989951031s" totalImagesPullingTime="0s" totalInitContainerRuntime="3.670419973s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:39:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:39:18.988928496 +0000 UTC m=+2779.493600448" watchObservedRunningTime="2026-10-30 22:39:18.989951031 +0000 UTC m=+2779.494622982"
Oct 30 22:39:19.871956 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:39:19.871926    4412 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw0-37d062d1dd/isvc-canary-promote-predictor-58d794bb6f-8j72d"
Oct 30 22:39:19.873196 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:39:19.873173    4412 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-37d062d1dd/isvc-canary-promote-predictor-58d794bb6f-8j72d" podUID="cf8b5667-ca48-460f-84d5-e6ee6e61c068" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.29:8080: connect: connection refused"
Oct 30 22:39:20.875193 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:39:20.875157    4412 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-37d062d1dd/isvc-canary-promote-predictor-58d794bb6f-8j72d" podUID="cf8b5667-ca48-460f-84d5-e6ee6e61c068" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.29:8080: connect: connection refused"
Oct 30 22:39:25.879962 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:39:25.879933    4412 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw0-37d062d1dd/isvc-canary-promote-predictor-58d794bb6f-8j72d"
Oct 30 22:39:25.880525 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:39:25.880501    4412 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-37d062d1dd/isvc-canary-promote-predictor-58d794bb6f-8j72d" podUID="cf8b5667-ca48-460f-84d5-e6ee6e61c068" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.29:8080: connect: connection refused"
Oct 30 22:39:34.447456 ip-10-0-131-202 kubenswrapper[4412]: E1030 22:39:34.447415    4412 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bkhmz" containerName="extractor"
Oct 30 22:39:35.884909 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:39:35.884864    4412 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-37d062d1dd/isvc-canary-promote-predictor-58d794bb6f-8j72d" podUID="cf8b5667-ca48-460f-84d5-e6ee6e61c068" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.29:8080: connect: connection refused"
Oct 30 22:39:45.880518 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:39:45.880477    4412 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-37d062d1dd/isvc-canary-promote-predictor-58d794bb6f-8j72d" podUID="cf8b5667-ca48-460f-84d5-e6ee6e61c068" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.29:8080: connect: connection refused"
Oct 30 22:39:55.881482 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:39:55.881441    4412 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-37d062d1dd/isvc-canary-promote-predictor-58d794bb6f-8j72d" podUID="cf8b5667-ca48-460f-84d5-e6ee6e61c068" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.29:8080: connect: connection refused"
Oct 30 22:39:59.913572 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:39:59.913546    4412 scope.go:122] "RemoveContainer" containerID="f8282d5c835211cf9269e95803a348f965c58477933d67a5e90107b30b70e786"
Oct 30 22:40:04.449005 ip-10-0-131-202 kubenswrapper[4412]: E1030 22:40:04.448962    4412 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bkhmz" containerName="extractor"
Oct 30 22:40:05.880565 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:40:05.880525    4412 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-37d062d1dd/isvc-canary-promote-predictor-58d794bb6f-8j72d" podUID="cf8b5667-ca48-460f-84d5-e6ee6e61c068" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.29:8080: connect: connection refused"
Oct 30 22:40:15.881448 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:40:15.881368    4412 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-37d062d1dd/isvc-canary-promote-predictor-58d794bb6f-8j72d" podUID="cf8b5667-ca48-460f-84d5-e6ee6e61c068" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.29:8080: connect: connection refused"
Oct 30 22:40:25.880548 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:40:25.880512    4412 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-37d062d1dd/isvc-canary-promote-predictor-58d794bb6f-8j72d" podUID="cf8b5667-ca48-460f-84d5-e6ee6e61c068" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.29:8080: connect: connection refused"
Oct 30 22:40:34.448360 ip-10-0-131-202 kubenswrapper[4412]: E1030 22:40:34.448314    4412 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bkhmz" containerName="extractor"
Oct 30 22:40:35.881825 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:40:35.881800    4412 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw0-37d062d1dd/isvc-canary-promote-predictor-58d794bb6f-8j72d"
Oct 30 22:40:51.883435 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:40:51.883401    4412 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw0-37d062d1dd/isvc-canary-promote-predictor-58d794bb6f-8j72d"]
Oct 30 22:40:51.883931 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:40:51.883627    4412 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw0-37d062d1dd/isvc-canary-promote-predictor-58d794bb6f-8j72d" podUID="cf8b5667-ca48-460f-84d5-e6ee6e61c068" containerName="kserve-container" containerID="cri-o://10bed3e9713eb9c2c43455d2f07411f56abfe71df9a716defea3e175418e305d" gracePeriod=30
Oct 30 22:40:51.883931 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:40:51.883667    4412 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw0-37d062d1dd/isvc-canary-promote-predictor-58d794bb6f-8j72d" podUID="cf8b5667-ca48-460f-84d5-e6ee6e61c068" containerName="kube-rbac-proxy" containerID="cri-o://a6c1476e9055709d57dbed4e59f826c259bc0539de176c4769b86a2b1035526c" gracePeriod=30
Oct 30 22:40:52.213933 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:40:52.213903    4412 generic.go:356] "Generic (PLEG): container finished" podID="cf8b5667-ca48-460f-84d5-e6ee6e61c068" containerID="a6c1476e9055709d57dbed4e59f826c259bc0539de176c4769b86a2b1035526c" exitCode=0
Oct 30 22:40:52.214094 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:40:52.213953    4412 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-37d062d1dd/isvc-canary-promote-predictor-58d794bb6f-8j72d" event={"ID":"cf8b5667-ca48-460f-84d5-e6ee6e61c068","Type":"ContainerDied","Data":"a6c1476e9055709d57dbed4e59f826c259bc0539de176c4769b86a2b1035526c"}
Oct 30 22:40:52.223919 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:40:52.223895    4412 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-gw0-37d062d1dd/isvc-canary-promote-v2-predictor-6d897ccb86-s489p"]
Oct 30 22:40:52.227045 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:40:52.227031    4412 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-37d062d1dd/isvc-canary-promote-v2-predictor-6d897ccb86-s489p"
Oct 30 22:40:52.280035 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:40:52.280012    4412 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw0-37d062d1dd\"/\"isvc-canary-promote-v2-predictor-serving-cert\""
Oct 30 22:40:52.291567 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:40:52.291542    4412 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw0-37d062d1dd/isvc-canary-promote-v2-predictor-6d897ccb86-s489p"]
Oct 30 22:40:52.338006 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:40:52.337978    4412 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/fe258e1a-bb23-4405-bb6f-6b993c02f2b3-kserve-provision-location\") pod \"isvc-canary-promote-v2-predictor-6d897ccb86-s489p\" (UID: \"fe258e1a-bb23-4405-bb6f-6b993c02f2b3\") " pod="e2e-gw0-37d062d1dd/isvc-canary-promote-v2-predictor-6d897ccb86-s489p"
Oct 30 22:40:52.338145 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:40:52.338009    4412 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/fe258e1a-bb23-4405-bb6f-6b993c02f2b3-cabundle-cert\") pod \"isvc-canary-promote-v2-predictor-6d897ccb86-s489p\" (UID: \"fe258e1a-bb23-4405-bb6f-6b993c02f2b3\") " pod="e2e-gw0-37d062d1dd/isvc-canary-promote-v2-predictor-6d897ccb86-s489p"
Oct 30 22:40:52.338145 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:40:52.338040    4412 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/fe258e1a-bb23-4405-bb6f-6b993c02f2b3-kube-rbac-proxy-sar-config\") pod \"isvc-canary-promote-v2-predictor-6d897ccb86-s489p\" (UID: \"fe258e1a-bb23-4405-bb6f-6b993c02f2b3\") " pod="e2e-gw0-37d062d1dd/isvc-canary-promote-v2-predictor-6d897ccb86-s489p"
Oct 30 22:40:52.338242 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:40:52.338209    4412 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/fe258e1a-bb23-4405-bb6f-6b993c02f2b3-proxy-tls\") pod \"isvc-canary-promote-v2-predictor-6d897ccb86-s489p\" (UID: \"fe258e1a-bb23-4405-bb6f-6b993c02f2b3\") " pod="e2e-gw0-37d062d1dd/isvc-canary-promote-v2-predictor-6d897ccb86-s489p"
Oct 30 22:40:52.338279 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:40:52.338252    4412 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hxtv2\" (UniqueName: \"kubernetes.io/projected/fe258e1a-bb23-4405-bb6f-6b993c02f2b3-kube-api-access-hxtv2\") pod \"isvc-canary-promote-v2-predictor-6d897ccb86-s489p\" (UID: \"fe258e1a-bb23-4405-bb6f-6b993c02f2b3\") " pod="e2e-gw0-37d062d1dd/isvc-canary-promote-v2-predictor-6d897ccb86-s489p"
Oct 30 22:40:52.439490 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:40:52.439463    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/fe258e1a-bb23-4405-bb6f-6b993c02f2b3-kube-rbac-proxy-sar-config\") pod \"isvc-canary-promote-v2-predictor-6d897ccb86-s489p\" (UID: \"fe258e1a-bb23-4405-bb6f-6b993c02f2b3\") " pod="e2e-gw0-37d062d1dd/isvc-canary-promote-v2-predictor-6d897ccb86-s489p"
Oct 30 22:40:52.439661 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:40:52.439507    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/fe258e1a-bb23-4405-bb6f-6b993c02f2b3-proxy-tls\") pod \"isvc-canary-promote-v2-predictor-6d897ccb86-s489p\" (UID: \"fe258e1a-bb23-4405-bb6f-6b993c02f2b3\") " pod="e2e-gw0-37d062d1dd/isvc-canary-promote-v2-predictor-6d897ccb86-s489p"
Oct 30 22:40:52.439661 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:40:52.439531    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hxtv2\" (UniqueName: \"kubernetes.io/projected/fe258e1a-bb23-4405-bb6f-6b993c02f2b3-kube-api-access-hxtv2\") pod \"isvc-canary-promote-v2-predictor-6d897ccb86-s489p\" (UID: \"fe258e1a-bb23-4405-bb6f-6b993c02f2b3\") " pod="e2e-gw0-37d062d1dd/isvc-canary-promote-v2-predictor-6d897ccb86-s489p"
Oct 30 22:40:52.439661 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:40:52.439567    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/fe258e1a-bb23-4405-bb6f-6b993c02f2b3-kserve-provision-location\") pod \"isvc-canary-promote-v2-predictor-6d897ccb86-s489p\" (UID: \"fe258e1a-bb23-4405-bb6f-6b993c02f2b3\") " pod="e2e-gw0-37d062d1dd/isvc-canary-promote-v2-predictor-6d897ccb86-s489p"
Oct 30 22:40:52.439661 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:40:52.439582    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/fe258e1a-bb23-4405-bb6f-6b993c02f2b3-cabundle-cert\") pod \"isvc-canary-promote-v2-predictor-6d897ccb86-s489p\" (UID: \"fe258e1a-bb23-4405-bb6f-6b993c02f2b3\") " pod="e2e-gw0-37d062d1dd/isvc-canary-promote-v2-predictor-6d897ccb86-s489p"
Oct 30 22:40:52.439992 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:40:52.439959    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/fe258e1a-bb23-4405-bb6f-6b993c02f2b3-kserve-provision-location\") pod \"isvc-canary-promote-v2-predictor-6d897ccb86-s489p\" (UID: \"fe258e1a-bb23-4405-bb6f-6b993c02f2b3\") " pod="e2e-gw0-37d062d1dd/isvc-canary-promote-v2-predictor-6d897ccb86-s489p"
Oct 30 22:40:52.440305 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:40:52.440242    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/fe258e1a-bb23-4405-bb6f-6b993c02f2b3-kube-rbac-proxy-sar-config\") pod \"isvc-canary-promote-v2-predictor-6d897ccb86-s489p\" (UID: \"fe258e1a-bb23-4405-bb6f-6b993c02f2b3\") " pod="e2e-gw0-37d062d1dd/isvc-canary-promote-v2-predictor-6d897ccb86-s489p"
Oct 30 22:40:52.440305 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:40:52.440248    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/fe258e1a-bb23-4405-bb6f-6b993c02f2b3-cabundle-cert\") pod \"isvc-canary-promote-v2-predictor-6d897ccb86-s489p\" (UID: \"fe258e1a-bb23-4405-bb6f-6b993c02f2b3\") " pod="e2e-gw0-37d062d1dd/isvc-canary-promote-v2-predictor-6d897ccb86-s489p"
Oct 30 22:40:52.442209 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:40:52.442185    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/fe258e1a-bb23-4405-bb6f-6b993c02f2b3-proxy-tls\") pod \"isvc-canary-promote-v2-predictor-6d897ccb86-s489p\" (UID: \"fe258e1a-bb23-4405-bb6f-6b993c02f2b3\") " pod="e2e-gw0-37d062d1dd/isvc-canary-promote-v2-predictor-6d897ccb86-s489p"
Oct 30 22:40:52.497227 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:40:52.497174    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hxtv2\" (UniqueName: \"kubernetes.io/projected/fe258e1a-bb23-4405-bb6f-6b993c02f2b3-kube-api-access-hxtv2\") pod \"isvc-canary-promote-v2-predictor-6d897ccb86-s489p\" (UID: \"fe258e1a-bb23-4405-bb6f-6b993c02f2b3\") " pod="e2e-gw0-37d062d1dd/isvc-canary-promote-v2-predictor-6d897ccb86-s489p"
Oct 30 22:40:52.547761 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:40:52.547737    4412 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-37d062d1dd/isvc-canary-promote-v2-predictor-6d897ccb86-s489p"
Oct 30 22:40:52.966495 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:40:52.966090    4412 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw0-37d062d1dd/isvc-canary-promote-v2-predictor-6d897ccb86-s489p"]
Oct 30 22:40:52.969255 ip-10-0-131-202 kubenswrapper[4412]: W1030 22:40:52.969219    4412 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfe258e1a_bb23_4405_bb6f_6b993c02f2b3.slice/crio-a6764e7bfc716b09535e0dcb0e62f6ad94c686aaae79b932fc3d0f9e20f1241a WatchSource:0}: Error finding container a6764e7bfc716b09535e0dcb0e62f6ad94c686aaae79b932fc3d0f9e20f1241a: Status 404 returned error can't find the container with id a6764e7bfc716b09535e0dcb0e62f6ad94c686aaae79b932fc3d0f9e20f1241a
Oct 30 22:40:53.220652 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:40:53.220567    4412 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-37d062d1dd/isvc-canary-promote-v2-predictor-6d897ccb86-s489p" event={"ID":"fe258e1a-bb23-4405-bb6f-6b993c02f2b3","Type":"ContainerStarted","Data":"2a7a33616925d9dd8b2904ec3ad8d0a18ff4faac067fe6d76e4b84b33cf5e130"}
Oct 30 22:40:53.220652 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:40:53.220628    4412 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-37d062d1dd/isvc-canary-promote-v2-predictor-6d897ccb86-s489p" event={"ID":"fe258e1a-bb23-4405-bb6f-6b993c02f2b3","Type":"ContainerStarted","Data":"a6764e7bfc716b09535e0dcb0e62f6ad94c686aaae79b932fc3d0f9e20f1241a"}
Oct 30 22:40:55.875963 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:40:55.875921    4412 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-37d062d1dd/isvc-canary-promote-predictor-58d794bb6f-8j72d" podUID="cf8b5667-ca48-460f-84d5-e6ee6e61c068" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.29:8643/healthz\": dial tcp 10.133.0.29:8643: connect: connection refused"
Oct 30 22:40:55.881282 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:40:55.881257    4412 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-37d062d1dd/isvc-canary-promote-predictor-58d794bb6f-8j72d" podUID="cf8b5667-ca48-460f-84d5-e6ee6e61c068" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.29:8080: connect: connection refused"
Oct 30 22:40:56.626453 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:40:56.626430    4412 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-37d062d1dd/isvc-canary-promote-predictor-58d794bb6f-8j72d"
Oct 30 22:40:56.777126 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:40:56.777097    4412 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/cf8b5667-ca48-460f-84d5-e6ee6e61c068-proxy-tls\" (UniqueName: \"kubernetes.io/secret/cf8b5667-ca48-460f-84d5-e6ee6e61c068-proxy-tls\") pod \"cf8b5667-ca48-460f-84d5-e6ee6e61c068\" (UID: \"cf8b5667-ca48-460f-84d5-e6ee6e61c068\") "
Oct 30 22:40:56.777328 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:40:56.777163    4412 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/cf8b5667-ca48-460f-84d5-e6ee6e61c068-cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/cf8b5667-ca48-460f-84d5-e6ee6e61c068-cabundle-cert\") pod \"cf8b5667-ca48-460f-84d5-e6ee6e61c068\" (UID: \"cf8b5667-ca48-460f-84d5-e6ee6e61c068\") "
Oct 30 22:40:56.777328 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:40:56.777193    4412 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/cf8b5667-ca48-460f-84d5-e6ee6e61c068-kube-api-access-6rhln\" (UniqueName: \"kubernetes.io/projected/cf8b5667-ca48-460f-84d5-e6ee6e61c068-kube-api-access-6rhln\") pod \"cf8b5667-ca48-460f-84d5-e6ee6e61c068\" (UID: \"cf8b5667-ca48-460f-84d5-e6ee6e61c068\") "
Oct 30 22:40:56.777328 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:40:56.777304    4412 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cf8b5667-ca48-460f-84d5-e6ee6e61c068-kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/cf8b5667-ca48-460f-84d5-e6ee6e61c068-kserve-provision-location\") pod \"cf8b5667-ca48-460f-84d5-e6ee6e61c068\" (UID: \"cf8b5667-ca48-460f-84d5-e6ee6e61c068\") "
Oct 30 22:40:56.777510 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:40:56.777376    4412 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/cf8b5667-ca48-460f-84d5-e6ee6e61c068-kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/cf8b5667-ca48-460f-84d5-e6ee6e61c068-kube-rbac-proxy-sar-config\") pod \"cf8b5667-ca48-460f-84d5-e6ee6e61c068\" (UID: \"cf8b5667-ca48-460f-84d5-e6ee6e61c068\") "
Oct 30 22:40:56.777577 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:40:56.777535    4412 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cf8b5667-ca48-460f-84d5-e6ee6e61c068-cabundle-cert" pod "cf8b5667-ca48-460f-84d5-e6ee6e61c068" (UID: "cf8b5667-ca48-460f-84d5-e6ee6e61c068"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:40:56.777636 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:40:56.777571    4412 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cf8b5667-ca48-460f-84d5-e6ee6e61c068-kserve-provision-location" pod "cf8b5667-ca48-460f-84d5-e6ee6e61c068" (UID: "cf8b5667-ca48-460f-84d5-e6ee6e61c068"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:40:56.777756 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:40:56.777702    4412 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/cf8b5667-ca48-460f-84d5-e6ee6e61c068-cabundle-cert\") on node \"ip-10-0-131-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:40:56.777756 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:40:56.777724    4412 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/cf8b5667-ca48-460f-84d5-e6ee6e61c068-kserve-provision-location\") on node \"ip-10-0-131-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:40:56.777756 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:40:56.777698    4412 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cf8b5667-ca48-460f-84d5-e6ee6e61c068-kube-rbac-proxy-sar-config" pod "cf8b5667-ca48-460f-84d5-e6ee6e61c068" (UID: "cf8b5667-ca48-460f-84d5-e6ee6e61c068"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:40:56.779536 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:40:56.779502    4412 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cf8b5667-ca48-460f-84d5-e6ee6e61c068-proxy-tls" pod "cf8b5667-ca48-460f-84d5-e6ee6e61c068" (UID: "cf8b5667-ca48-460f-84d5-e6ee6e61c068"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:40:56.779623 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:40:56.779572    4412 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cf8b5667-ca48-460f-84d5-e6ee6e61c068-kube-api-access-6rhln" pod "cf8b5667-ca48-460f-84d5-e6ee6e61c068" (UID: "cf8b5667-ca48-460f-84d5-e6ee6e61c068"). InnerVolumeSpecName "kube-api-access-6rhln". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:40:56.878458 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:40:56.878425    4412 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/cf8b5667-ca48-460f-84d5-e6ee6e61c068-proxy-tls\") on node \"ip-10-0-131-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:40:56.878458 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:40:56.878453    4412 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6rhln\" (UniqueName: \"kubernetes.io/projected/cf8b5667-ca48-460f-84d5-e6ee6e61c068-kube-api-access-6rhln\") on node \"ip-10-0-131-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:40:56.878843 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:40:56.878467    4412 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/cf8b5667-ca48-460f-84d5-e6ee6e61c068-kube-rbac-proxy-sar-config\") on node \"ip-10-0-131-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:40:57.236455 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:40:57.236418    4412 generic.go:356] "Generic (PLEG): container finished" podID="cf8b5667-ca48-460f-84d5-e6ee6e61c068" containerID="10bed3e9713eb9c2c43455d2f07411f56abfe71df9a716defea3e175418e305d" exitCode=0
Oct 30 22:40:57.236634 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:40:57.236488    4412 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-37d062d1dd/isvc-canary-promote-predictor-58d794bb6f-8j72d"
Oct 30 22:40:57.236634 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:40:57.236502    4412 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-37d062d1dd/isvc-canary-promote-predictor-58d794bb6f-8j72d" event={"ID":"cf8b5667-ca48-460f-84d5-e6ee6e61c068","Type":"ContainerDied","Data":"10bed3e9713eb9c2c43455d2f07411f56abfe71df9a716defea3e175418e305d"}
Oct 30 22:40:57.236634 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:40:57.236573    4412 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-37d062d1dd/isvc-canary-promote-predictor-58d794bb6f-8j72d" event={"ID":"cf8b5667-ca48-460f-84d5-e6ee6e61c068","Type":"ContainerDied","Data":"6f4d8f7ccca49a991db06aab7a453af4d515cd9aacb6b90015a8f16a9c970330"}
Oct 30 22:40:57.236634 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:40:57.236610    4412 scope.go:122] "RemoveContainer" containerID="a6c1476e9055709d57dbed4e59f826c259bc0539de176c4769b86a2b1035526c"
Oct 30 22:40:57.238097 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:40:57.238072    4412 generic.go:356] "Generic (PLEG): container finished" podID="fe258e1a-bb23-4405-bb6f-6b993c02f2b3" containerID="2a7a33616925d9dd8b2904ec3ad8d0a18ff4faac067fe6d76e4b84b33cf5e130" exitCode=0
Oct 30 22:40:57.238294 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:40:57.238159    4412 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-37d062d1dd/isvc-canary-promote-v2-predictor-6d897ccb86-s489p" event={"ID":"fe258e1a-bb23-4405-bb6f-6b993c02f2b3","Type":"ContainerDied","Data":"2a7a33616925d9dd8b2904ec3ad8d0a18ff4faac067fe6d76e4b84b33cf5e130"}
Oct 30 22:40:57.248920 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:40:57.248901    4412 scope.go:122] "RemoveContainer" containerID="10bed3e9713eb9c2c43455d2f07411f56abfe71df9a716defea3e175418e305d"
Oct 30 22:40:57.256967 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:40:57.256948    4412 scope.go:122] "RemoveContainer" containerID="2647831515c72c725f7248e0d324334530e283a28f2444530da0903788d66569"
Oct 30 22:40:57.272688 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:40:57.272666    4412 scope.go:122] "RemoveContainer" containerID="a6c1476e9055709d57dbed4e59f826c259bc0539de176c4769b86a2b1035526c"
Oct 30 22:40:57.272961 ip-10-0-131-202 kubenswrapper[4412]: E1030 22:40:57.272939    4412 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a6c1476e9055709d57dbed4e59f826c259bc0539de176c4769b86a2b1035526c\": container with ID starting with a6c1476e9055709d57dbed4e59f826c259bc0539de176c4769b86a2b1035526c not found: ID does not exist" containerID="a6c1476e9055709d57dbed4e59f826c259bc0539de176c4769b86a2b1035526c"
Oct 30 22:40:57.273018 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:40:57.272974    4412 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a6c1476e9055709d57dbed4e59f826c259bc0539de176c4769b86a2b1035526c"} err="failed to get container status \"a6c1476e9055709d57dbed4e59f826c259bc0539de176c4769b86a2b1035526c\": rpc error: code = NotFound desc = could not find container \"a6c1476e9055709d57dbed4e59f826c259bc0539de176c4769b86a2b1035526c\": container with ID starting with a6c1476e9055709d57dbed4e59f826c259bc0539de176c4769b86a2b1035526c not found: ID does not exist"
Oct 30 22:40:57.273018 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:40:57.272996    4412 scope.go:122] "RemoveContainer" containerID="10bed3e9713eb9c2c43455d2f07411f56abfe71df9a716defea3e175418e305d"
Oct 30 22:40:57.273277 ip-10-0-131-202 kubenswrapper[4412]: E1030 22:40:57.273258    4412 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"10bed3e9713eb9c2c43455d2f07411f56abfe71df9a716defea3e175418e305d\": container with ID starting with 10bed3e9713eb9c2c43455d2f07411f56abfe71df9a716defea3e175418e305d not found: ID does not exist" containerID="10bed3e9713eb9c2c43455d2f07411f56abfe71df9a716defea3e175418e305d"
Oct 30 22:40:57.273339 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:40:57.273284    4412 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"10bed3e9713eb9c2c43455d2f07411f56abfe71df9a716defea3e175418e305d"} err="failed to get container status \"10bed3e9713eb9c2c43455d2f07411f56abfe71df9a716defea3e175418e305d\": rpc error: code = NotFound desc = could not find container \"10bed3e9713eb9c2c43455d2f07411f56abfe71df9a716defea3e175418e305d\": container with ID starting with 10bed3e9713eb9c2c43455d2f07411f56abfe71df9a716defea3e175418e305d not found: ID does not exist"
Oct 30 22:40:57.273339 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:40:57.273300    4412 scope.go:122] "RemoveContainer" containerID="2647831515c72c725f7248e0d324334530e283a28f2444530da0903788d66569"
Oct 30 22:40:57.273588 ip-10-0-131-202 kubenswrapper[4412]: E1030 22:40:57.273572    4412 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2647831515c72c725f7248e0d324334530e283a28f2444530da0903788d66569\": container with ID starting with 2647831515c72c725f7248e0d324334530e283a28f2444530da0903788d66569 not found: ID does not exist" containerID="2647831515c72c725f7248e0d324334530e283a28f2444530da0903788d66569"
Oct 30 22:40:57.273631 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:40:57.273594    4412 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2647831515c72c725f7248e0d324334530e283a28f2444530da0903788d66569"} err="failed to get container status \"2647831515c72c725f7248e0d324334530e283a28f2444530da0903788d66569\": rpc error: code = NotFound desc = could not find container \"2647831515c72c725f7248e0d324334530e283a28f2444530da0903788d66569\": container with ID starting with 2647831515c72c725f7248e0d324334530e283a28f2444530da0903788d66569 not found: ID does not exist"
Oct 30 22:40:57.401652 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:40:57.401625    4412 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw0-37d062d1dd/isvc-canary-promote-predictor-58d794bb6f-8j72d"]
Oct 30 22:40:57.452035 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:40:57.452010    4412 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-gw0-37d062d1dd/isvc-canary-promote-predictor-58d794bb6f-8j72d"]
Oct 30 22:40:57.626157 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:40:57.626070    4412 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="cf8b5667-ca48-460f-84d5-e6ee6e61c068" path="/var/lib/kubelet/pods/cf8b5667-ca48-460f-84d5-e6ee6e61c068/volumes"
Oct 30 22:40:58.245062 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:40:58.245033    4412 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-37d062d1dd/isvc-canary-promote-v2-predictor-6d897ccb86-s489p" event={"ID":"fe258e1a-bb23-4405-bb6f-6b993c02f2b3","Type":"ContainerStarted","Data":"9df35da1b5f769248c0de7ea23f0a57495bc6f05876c21ab42efa90dc169a276"}
Oct 30 22:40:58.245458 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:40:58.245085    4412 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-37d062d1dd/isvc-canary-promote-v2-predictor-6d897ccb86-s489p" event={"ID":"fe258e1a-bb23-4405-bb6f-6b993c02f2b3","Type":"ContainerStarted","Data":"5412c56da258d132144ad48dcb5988b474aedc5d4cb79fee8cf87c44f006f929"}
Oct 30 22:40:58.245458 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:40:58.245362    4412 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw0-37d062d1dd/isvc-canary-promote-v2-predictor-6d897ccb86-s489p"
Oct 30 22:40:58.245458 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:40:58.245413    4412 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw0-37d062d1dd/isvc-canary-promote-v2-predictor-6d897ccb86-s489p"
Oct 30 22:40:58.246666 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:40:58.246646    4412 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-37d062d1dd/isvc-canary-promote-v2-predictor-6d897ccb86-s489p" podUID="fe258e1a-bb23-4405-bb6f-6b993c02f2b3" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.30:8080: connect: connection refused"
Oct 30 22:40:58.367868 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:40:58.367825    4412 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-gw0-37d062d1dd/isvc-canary-promote-v2-predictor-6d897ccb86-s489p" podStartSLOduration=2.357195211 podStartE2EDuration="6.367812227s" totalImagesPullingTime="0s" totalInitContainerRuntime="4.010617016s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:40:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:40:58.366795189 +0000 UTC m=+2878.871467141" watchObservedRunningTime="2026-10-30 22:40:58.367812227 +0000 UTC m=+2878.872484173"
Oct 30 22:40:59.249732 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:40:59.249695    4412 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-37d062d1dd/isvc-canary-promote-v2-predictor-6d897ccb86-s489p" podUID="fe258e1a-bb23-4405-bb6f-6b993c02f2b3" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.30:8080: connect: connection refused"
Oct 30 22:40:59.927486 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:40:59.927458    4412 scope.go:122] "RemoveContainer" containerID="5b201c378cdbad41f96f61f06bc3ad629080945fee98d1b482ccd49c6f51c849"
Oct 30 22:40:59.935150 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:40:59.935136    4412 scope.go:122] "RemoveContainer" containerID="770898c6cb04328bf0a0c02e4e6d7ebed83d6c09fcc8dc2fd827f0065de49fc3"
Oct 30 22:41:04.254763 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:41:04.254732    4412 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw0-37d062d1dd/isvc-canary-promote-v2-predictor-6d897ccb86-s489p"
Oct 30 22:41:04.255266 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:41:04.255239    4412 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-37d062d1dd/isvc-canary-promote-v2-predictor-6d897ccb86-s489p" podUID="fe258e1a-bb23-4405-bb6f-6b993c02f2b3" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.30:8080: connect: connection refused"
Oct 30 22:41:04.449448 ip-10-0-131-202 kubenswrapper[4412]: E1030 22:41:04.449413    4412 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bkhmz" containerName="extractor"
Oct 30 22:41:14.256041 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:41:14.256001    4412 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-37d062d1dd/isvc-canary-promote-v2-predictor-6d897ccb86-s489p" podUID="fe258e1a-bb23-4405-bb6f-6b993c02f2b3" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.30:8080: connect: connection refused"
Oct 30 22:41:24.255193 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:41:24.255156    4412 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-37d062d1dd/isvc-canary-promote-v2-predictor-6d897ccb86-s489p" podUID="fe258e1a-bb23-4405-bb6f-6b993c02f2b3" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.30:8080: connect: connection refused"
Oct 30 22:41:34.255734 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:41:34.255691    4412 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-37d062d1dd/isvc-canary-promote-v2-predictor-6d897ccb86-s489p" podUID="fe258e1a-bb23-4405-bb6f-6b993c02f2b3" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.30:8080: connect: connection refused"
Oct 30 22:41:34.450219 ip-10-0-131-202 kubenswrapper[4412]: E1030 22:41:34.450186    4412 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bkhmz" containerName="extractor"
Oct 30 22:41:44.255395 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:41:44.255313    4412 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-37d062d1dd/isvc-canary-promote-v2-predictor-6d897ccb86-s489p" podUID="fe258e1a-bb23-4405-bb6f-6b993c02f2b3" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.30:8080: connect: connection refused"
Oct 30 22:41:54.255744 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:41:54.255702    4412 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-37d062d1dd/isvc-canary-promote-v2-predictor-6d897ccb86-s489p" podUID="fe258e1a-bb23-4405-bb6f-6b993c02f2b3" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.30:8080: connect: connection refused"
Oct 30 22:42:04.255182 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:42:04.255112    4412 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-37d062d1dd/isvc-canary-promote-v2-predictor-6d897ccb86-s489p" podUID="fe258e1a-bb23-4405-bb6f-6b993c02f2b3" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.30:8080: connect: connection refused"
Oct 30 22:42:04.448600 ip-10-0-131-202 kubenswrapper[4412]: E1030 22:42:04.448559    4412 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bkhmz" containerName="extractor"
Oct 30 22:42:14.256254 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:42:14.256223    4412 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw0-37d062d1dd/isvc-canary-promote-v2-predictor-6d897ccb86-s489p"
Oct 30 22:42:23.900022 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:42:23.899977    4412 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw0-37d062d1dd/isvc-canary-promote-v2-predictor-6d897ccb86-s489p"]
Oct 30 22:42:23.900456 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:42:23.900249    4412 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw0-37d062d1dd/isvc-canary-promote-v2-predictor-6d897ccb86-s489p" podUID="fe258e1a-bb23-4405-bb6f-6b993c02f2b3" containerName="kserve-container" containerID="cri-o://5412c56da258d132144ad48dcb5988b474aedc5d4cb79fee8cf87c44f006f929" gracePeriod=30
Oct 30 22:42:23.900456 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:42:23.900281    4412 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw0-37d062d1dd/isvc-canary-promote-v2-predictor-6d897ccb86-s489p" podUID="fe258e1a-bb23-4405-bb6f-6b993c02f2b3" containerName="kube-rbac-proxy" containerID="cri-o://9df35da1b5f769248c0de7ea23f0a57495bc6f05876c21ab42efa90dc169a276" gracePeriod=30
Oct 30 22:42:24.250556 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:42:24.250516    4412 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-37d062d1dd/isvc-canary-promote-v2-predictor-6d897ccb86-s489p" podUID="fe258e1a-bb23-4405-bb6f-6b993c02f2b3" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.30:8643/healthz\": dial tcp 10.133.0.30:8643: connect: connection refused"
Oct 30 22:42:24.255883 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:42:24.255857    4412 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-37d062d1dd/isvc-canary-promote-v2-predictor-6d897ccb86-s489p" podUID="fe258e1a-bb23-4405-bb6f-6b993c02f2b3" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.30:8080: connect: connection refused"
Oct 30 22:42:24.558667 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:42:24.558590    4412 generic.go:356] "Generic (PLEG): container finished" podID="fe258e1a-bb23-4405-bb6f-6b993c02f2b3" containerID="9df35da1b5f769248c0de7ea23f0a57495bc6f05876c21ab42efa90dc169a276" exitCode=0
Oct 30 22:42:24.558815 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:42:24.558667    4412 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-37d062d1dd/isvc-canary-promote-v2-predictor-6d897ccb86-s489p" event={"ID":"fe258e1a-bb23-4405-bb6f-6b993c02f2b3","Type":"ContainerDied","Data":"9df35da1b5f769248c0de7ea23f0a57495bc6f05876c21ab42efa90dc169a276"}
Oct 30 22:42:28.574600 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:42:28.574570    4412 generic.go:356] "Generic (PLEG): container finished" podID="fe258e1a-bb23-4405-bb6f-6b993c02f2b3" containerID="5412c56da258d132144ad48dcb5988b474aedc5d4cb79fee8cf87c44f006f929" exitCode=0
Oct 30 22:42:28.574942 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:42:28.574642    4412 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-37d062d1dd/isvc-canary-promote-v2-predictor-6d897ccb86-s489p" event={"ID":"fe258e1a-bb23-4405-bb6f-6b993c02f2b3","Type":"ContainerDied","Data":"5412c56da258d132144ad48dcb5988b474aedc5d4cb79fee8cf87c44f006f929"}
Oct 30 22:42:28.642605 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:42:28.642580    4412 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-37d062d1dd/isvc-canary-promote-v2-predictor-6d897ccb86-s489p"
Oct 30 22:42:28.685305 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:42:28.685282    4412 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fe258e1a-bb23-4405-bb6f-6b993c02f2b3-kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/fe258e1a-bb23-4405-bb6f-6b993c02f2b3-kserve-provision-location\") pod \"fe258e1a-bb23-4405-bb6f-6b993c02f2b3\" (UID: \"fe258e1a-bb23-4405-bb6f-6b993c02f2b3\") "
Oct 30 22:42:28.685414 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:42:28.685315    4412 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/fe258e1a-bb23-4405-bb6f-6b993c02f2b3-proxy-tls\" (UniqueName: \"kubernetes.io/secret/fe258e1a-bb23-4405-bb6f-6b993c02f2b3-proxy-tls\") pod \"fe258e1a-bb23-4405-bb6f-6b993c02f2b3\" (UID: \"fe258e1a-bb23-4405-bb6f-6b993c02f2b3\") "
Oct 30 22:42:28.685414 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:42:28.685340    4412 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/fe258e1a-bb23-4405-bb6f-6b993c02f2b3-cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/fe258e1a-bb23-4405-bb6f-6b993c02f2b3-cabundle-cert\") pod \"fe258e1a-bb23-4405-bb6f-6b993c02f2b3\" (UID: \"fe258e1a-bb23-4405-bb6f-6b993c02f2b3\") "
Oct 30 22:42:28.685414 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:42:28.685359    4412 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/fe258e1a-bb23-4405-bb6f-6b993c02f2b3-kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/fe258e1a-bb23-4405-bb6f-6b993c02f2b3-kube-rbac-proxy-sar-config\") pod \"fe258e1a-bb23-4405-bb6f-6b993c02f2b3\" (UID: \"fe258e1a-bb23-4405-bb6f-6b993c02f2b3\") "
Oct 30 22:42:28.685535 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:42:28.685501    4412 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fe258e1a-bb23-4405-bb6f-6b993c02f2b3-kube-api-access-hxtv2\" (UniqueName: \"kubernetes.io/projected/fe258e1a-bb23-4405-bb6f-6b993c02f2b3-kube-api-access-hxtv2\") pod \"fe258e1a-bb23-4405-bb6f-6b993c02f2b3\" (UID: \"fe258e1a-bb23-4405-bb6f-6b993c02f2b3\") "
Oct 30 22:42:28.685643 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:42:28.685622    4412 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fe258e1a-bb23-4405-bb6f-6b993c02f2b3-kserve-provision-location" pod "fe258e1a-bb23-4405-bb6f-6b993c02f2b3" (UID: "fe258e1a-bb23-4405-bb6f-6b993c02f2b3"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:42:28.685711 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:42:28.685699    4412 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fe258e1a-bb23-4405-bb6f-6b993c02f2b3-kube-rbac-proxy-sar-config" pod "fe258e1a-bb23-4405-bb6f-6b993c02f2b3" (UID: "fe258e1a-bb23-4405-bb6f-6b993c02f2b3"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:42:28.685766 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:42:28.685697    4412 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fe258e1a-bb23-4405-bb6f-6b993c02f2b3-cabundle-cert" pod "fe258e1a-bb23-4405-bb6f-6b993c02f2b3" (UID: "fe258e1a-bb23-4405-bb6f-6b993c02f2b3"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:42:28.685828 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:42:28.685800    4412 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/fe258e1a-bb23-4405-bb6f-6b993c02f2b3-cabundle-cert\") on node \"ip-10-0-131-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:42:28.685828 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:42:28.685816    4412 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/fe258e1a-bb23-4405-bb6f-6b993c02f2b3-kube-rbac-proxy-sar-config\") on node \"ip-10-0-131-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:42:28.685828 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:42:28.685826    4412 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/fe258e1a-bb23-4405-bb6f-6b993c02f2b3-kserve-provision-location\") on node \"ip-10-0-131-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:42:28.687554 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:42:28.687538    4412 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fe258e1a-bb23-4405-bb6f-6b993c02f2b3-kube-api-access-hxtv2" pod "fe258e1a-bb23-4405-bb6f-6b993c02f2b3" (UID: "fe258e1a-bb23-4405-bb6f-6b993c02f2b3"). InnerVolumeSpecName "kube-api-access-hxtv2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:42:28.687655 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:42:28.687638    4412 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fe258e1a-bb23-4405-bb6f-6b993c02f2b3-proxy-tls" pod "fe258e1a-bb23-4405-bb6f-6b993c02f2b3" (UID: "fe258e1a-bb23-4405-bb6f-6b993c02f2b3"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:42:28.786415 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:42:28.786382    4412 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hxtv2\" (UniqueName: \"kubernetes.io/projected/fe258e1a-bb23-4405-bb6f-6b993c02f2b3-kube-api-access-hxtv2\") on node \"ip-10-0-131-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:42:28.786415 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:42:28.786407    4412 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/fe258e1a-bb23-4405-bb6f-6b993c02f2b3-proxy-tls\") on node \"ip-10-0-131-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:42:29.580099 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:42:29.580062    4412 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-37d062d1dd/isvc-canary-promote-v2-predictor-6d897ccb86-s489p" event={"ID":"fe258e1a-bb23-4405-bb6f-6b993c02f2b3","Type":"ContainerDied","Data":"a6764e7bfc716b09535e0dcb0e62f6ad94c686aaae79b932fc3d0f9e20f1241a"}
Oct 30 22:42:29.580099 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:42:29.580085    4412 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-37d062d1dd/isvc-canary-promote-v2-predictor-6d897ccb86-s489p"
Oct 30 22:42:29.580668 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:42:29.580145    4412 scope.go:122] "RemoveContainer" containerID="9df35da1b5f769248c0de7ea23f0a57495bc6f05876c21ab42efa90dc169a276"
Oct 30 22:42:29.592635 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:42:29.592103    4412 scope.go:122] "RemoveContainer" containerID="5412c56da258d132144ad48dcb5988b474aedc5d4cb79fee8cf87c44f006f929"
Oct 30 22:42:29.599362 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:42:29.599342    4412 scope.go:122] "RemoveContainer" containerID="2a7a33616925d9dd8b2904ec3ad8d0a18ff4faac067fe6d76e4b84b33cf5e130"
Oct 30 22:42:29.801508 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:42:29.801475    4412 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw0-37d062d1dd/isvc-canary-promote-v2-predictor-6d897ccb86-s489p"]
Oct 30 22:42:29.851973 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:42:29.851905    4412 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-gw0-37d062d1dd/isvc-canary-promote-v2-predictor-6d897ccb86-s489p"]
Oct 30 22:42:31.625844 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:42:31.625814    4412 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="fe258e1a-bb23-4405-bb6f-6b993c02f2b3" path="/var/lib/kubelet/pods/fe258e1a-bb23-4405-bb6f-6b993c02f2b3/volumes"
Oct 30 22:42:33.940769 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:42:33.940737    4412 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-gw0-37d062d1dd/isvc-canary-rollback-predictor-78cd59d4dc-ptxth"]
Oct 30 22:42:33.941186 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:42:33.941027    4412 cpu_manager.go:414] "containerMap: removing container" podUID="fe258e1a-bb23-4405-bb6f-6b993c02f2b3" containerName="kserve-container"
Oct 30 22:42:33.941186 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:42:33.941037    4412 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="fe258e1a-bb23-4405-bb6f-6b993c02f2b3" containerName="kserve-container"
Oct 30 22:42:33.941186 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:42:33.941047    4412 cpu_manager.go:414] "containerMap: removing container" podUID="cf8b5667-ca48-460f-84d5-e6ee6e61c068" containerName="kserve-container"
Oct 30 22:42:33.941186 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:42:33.941052    4412 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="cf8b5667-ca48-460f-84d5-e6ee6e61c068" containerName="kserve-container"
Oct 30 22:42:33.941186 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:42:33.941060    4412 cpu_manager.go:414] "containerMap: removing container" podUID="cf8b5667-ca48-460f-84d5-e6ee6e61c068" containerName="storage-initializer"
Oct 30 22:42:33.941186 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:42:33.941065    4412 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="cf8b5667-ca48-460f-84d5-e6ee6e61c068" containerName="storage-initializer"
Oct 30 22:42:33.941186 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:42:33.941071    4412 cpu_manager.go:414] "containerMap: removing container" podUID="cf8b5667-ca48-460f-84d5-e6ee6e61c068" containerName="kube-rbac-proxy"
Oct 30 22:42:33.941186 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:42:33.941075    4412 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="cf8b5667-ca48-460f-84d5-e6ee6e61c068" containerName="kube-rbac-proxy"
Oct 30 22:42:33.941186 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:42:33.941088    4412 cpu_manager.go:414] "containerMap: removing container" podUID="fe258e1a-bb23-4405-bb6f-6b993c02f2b3" containerName="kube-rbac-proxy"
Oct 30 22:42:33.941186 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:42:33.941093    4412 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="fe258e1a-bb23-4405-bb6f-6b993c02f2b3" containerName="kube-rbac-proxy"
Oct 30 22:42:33.941186 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:42:33.941103    4412 cpu_manager.go:414] "containerMap: removing container" podUID="fe258e1a-bb23-4405-bb6f-6b993c02f2b3" containerName="storage-initializer"
Oct 30 22:42:33.941186 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:42:33.941108    4412 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="fe258e1a-bb23-4405-bb6f-6b993c02f2b3" containerName="storage-initializer"
Oct 30 22:42:33.941186 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:42:33.941169    4412 memory_manager.go:365] "RemoveStaleState removing state" podUID="fe258e1a-bb23-4405-bb6f-6b993c02f2b3" containerName="kserve-container"
Oct 30 22:42:33.941186 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:42:33.941178    4412 memory_manager.go:365] "RemoveStaleState removing state" podUID="cf8b5667-ca48-460f-84d5-e6ee6e61c068" containerName="kserve-container"
Oct 30 22:42:33.941186 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:42:33.941183    4412 memory_manager.go:365] "RemoveStaleState removing state" podUID="cf8b5667-ca48-460f-84d5-e6ee6e61c068" containerName="kube-rbac-proxy"
Oct 30 22:42:33.941186 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:42:33.941190    4412 memory_manager.go:365] "RemoveStaleState removing state" podUID="fe258e1a-bb23-4405-bb6f-6b993c02f2b3" containerName="kube-rbac-proxy"
Oct 30 22:42:33.944211 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:42:33.944196    4412 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-37d062d1dd/isvc-canary-rollback-predictor-78cd59d4dc-ptxth"
Oct 30 22:42:33.998084 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:42:33.998060    4412 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw0-37d062d1dd\"/\"kube-root-ca.crt\""
Oct 30 22:42:34.026529 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:42:34.026503    4412 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/a6d6b87d-3f49-4f74-9fc8-d9bcae4de021-cabundle-cert\") pod \"isvc-canary-rollback-predictor-78cd59d4dc-ptxth\" (UID: \"a6d6b87d-3f49-4f74-9fc8-d9bcae4de021\") " pod="e2e-gw0-37d062d1dd/isvc-canary-rollback-predictor-78cd59d4dc-ptxth"
Oct 30 22:42:34.026649 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:42:34.026534    4412 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/a6d6b87d-3f49-4f74-9fc8-d9bcae4de021-proxy-tls\") pod \"isvc-canary-rollback-predictor-78cd59d4dc-ptxth\" (UID: \"a6d6b87d-3f49-4f74-9fc8-d9bcae4de021\") " pod="e2e-gw0-37d062d1dd/isvc-canary-rollback-predictor-78cd59d4dc-ptxth"
Oct 30 22:42:34.026649 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:42:34.026554    4412 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/a6d6b87d-3f49-4f74-9fc8-d9bcae4de021-kube-rbac-proxy-sar-config\") pod \"isvc-canary-rollback-predictor-78cd59d4dc-ptxth\" (UID: \"a6d6b87d-3f49-4f74-9fc8-d9bcae4de021\") " pod="e2e-gw0-37d062d1dd/isvc-canary-rollback-predictor-78cd59d4dc-ptxth"
Oct 30 22:42:34.026649 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:42:34.026579    4412 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jz5kh\" (UniqueName: \"kubernetes.io/projected/a6d6b87d-3f49-4f74-9fc8-d9bcae4de021-kube-api-access-jz5kh\") pod \"isvc-canary-rollback-predictor-78cd59d4dc-ptxth\" (UID: \"a6d6b87d-3f49-4f74-9fc8-d9bcae4de021\") " pod="e2e-gw0-37d062d1dd/isvc-canary-rollback-predictor-78cd59d4dc-ptxth"
Oct 30 22:42:34.026753 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:42:34.026656    4412 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/a6d6b87d-3f49-4f74-9fc8-d9bcae4de021-kserve-provision-location\") pod \"isvc-canary-rollback-predictor-78cd59d4dc-ptxth\" (UID: \"a6d6b87d-3f49-4f74-9fc8-d9bcae4de021\") " pod="e2e-gw0-37d062d1dd/isvc-canary-rollback-predictor-78cd59d4dc-ptxth"
Oct 30 22:42:34.048551 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:42:34.048516    4412 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw0-37d062d1dd\"/\"openshift-service-ca.crt\""
Oct 30 22:42:34.048680 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:42:34.048660    4412 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw0-37d062d1dd\"/\"default-dockercfg-4v4s8\""
Oct 30 22:42:34.048744 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:42:34.048714    4412 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw0-37d062d1dd\"/\"seaweedfs-s3-creds\""
Oct 30 22:42:34.048793 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:42:34.048776    4412 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw0-37d062d1dd\"/\"odh-kserve-custom-ca-bundle\""
Oct 30 22:42:34.048852 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:42:34.048802    4412 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw0-37d062d1dd\"/\"isvc-canary-rollback-kube-rbac-proxy-sar-config\""
Oct 30 22:42:34.048903 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:42:34.048861    4412 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw0-37d062d1dd\"/\"isvc-canary-rollback-predictor-serving-cert\""
Oct 30 22:42:34.050213 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:42:34.050193    4412 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw0-37d062d1dd/isvc-canary-rollback-predictor-78cd59d4dc-ptxth"]
Oct 30 22:42:34.127745 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:42:34.127651    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jz5kh\" (UniqueName: \"kubernetes.io/projected/a6d6b87d-3f49-4f74-9fc8-d9bcae4de021-kube-api-access-jz5kh\") pod \"isvc-canary-rollback-predictor-78cd59d4dc-ptxth\" (UID: \"a6d6b87d-3f49-4f74-9fc8-d9bcae4de021\") " pod="e2e-gw0-37d062d1dd/isvc-canary-rollback-predictor-78cd59d4dc-ptxth"
Oct 30 22:42:34.127834 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:42:34.127768    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/a6d6b87d-3f49-4f74-9fc8-d9bcae4de021-kserve-provision-location\") pod \"isvc-canary-rollback-predictor-78cd59d4dc-ptxth\" (UID: \"a6d6b87d-3f49-4f74-9fc8-d9bcae4de021\") " pod="e2e-gw0-37d062d1dd/isvc-canary-rollback-predictor-78cd59d4dc-ptxth"
Oct 30 22:42:34.127890 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:42:34.127874    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/a6d6b87d-3f49-4f74-9fc8-d9bcae4de021-cabundle-cert\") pod \"isvc-canary-rollback-predictor-78cd59d4dc-ptxth\" (UID: \"a6d6b87d-3f49-4f74-9fc8-d9bcae4de021\") " pod="e2e-gw0-37d062d1dd/isvc-canary-rollback-predictor-78cd59d4dc-ptxth"
Oct 30 22:42:34.127926 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:42:34.127912    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/a6d6b87d-3f49-4f74-9fc8-d9bcae4de021-proxy-tls\") pod \"isvc-canary-rollback-predictor-78cd59d4dc-ptxth\" (UID: \"a6d6b87d-3f49-4f74-9fc8-d9bcae4de021\") " pod="e2e-gw0-37d062d1dd/isvc-canary-rollback-predictor-78cd59d4dc-ptxth"
Oct 30 22:42:34.127960 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:42:34.127932    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/a6d6b87d-3f49-4f74-9fc8-d9bcae4de021-kube-rbac-proxy-sar-config\") pod \"isvc-canary-rollback-predictor-78cd59d4dc-ptxth\" (UID: \"a6d6b87d-3f49-4f74-9fc8-d9bcae4de021\") " pod="e2e-gw0-37d062d1dd/isvc-canary-rollback-predictor-78cd59d4dc-ptxth"
Oct 30 22:42:34.128272 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:42:34.128250    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/a6d6b87d-3f49-4f74-9fc8-d9bcae4de021-kserve-provision-location\") pod \"isvc-canary-rollback-predictor-78cd59d4dc-ptxth\" (UID: \"a6d6b87d-3f49-4f74-9fc8-d9bcae4de021\") " pod="e2e-gw0-37d062d1dd/isvc-canary-rollback-predictor-78cd59d4dc-ptxth"
Oct 30 22:42:34.128518 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:42:34.128501    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/a6d6b87d-3f49-4f74-9fc8-d9bcae4de021-cabundle-cert\") pod \"isvc-canary-rollback-predictor-78cd59d4dc-ptxth\" (UID: \"a6d6b87d-3f49-4f74-9fc8-d9bcae4de021\") " pod="e2e-gw0-37d062d1dd/isvc-canary-rollback-predictor-78cd59d4dc-ptxth"
Oct 30 22:42:34.128612 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:42:34.128594    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/a6d6b87d-3f49-4f74-9fc8-d9bcae4de021-kube-rbac-proxy-sar-config\") pod \"isvc-canary-rollback-predictor-78cd59d4dc-ptxth\" (UID: \"a6d6b87d-3f49-4f74-9fc8-d9bcae4de021\") " pod="e2e-gw0-37d062d1dd/isvc-canary-rollback-predictor-78cd59d4dc-ptxth"
Oct 30 22:42:34.130511 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:42:34.130492    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/a6d6b87d-3f49-4f74-9fc8-d9bcae4de021-proxy-tls\") pod \"isvc-canary-rollback-predictor-78cd59d4dc-ptxth\" (UID: \"a6d6b87d-3f49-4f74-9fc8-d9bcae4de021\") " pod="e2e-gw0-37d062d1dd/isvc-canary-rollback-predictor-78cd59d4dc-ptxth"
Oct 30 22:42:34.338266 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:42:34.338242    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jz5kh\" (UniqueName: \"kubernetes.io/projected/a6d6b87d-3f49-4f74-9fc8-d9bcae4de021-kube-api-access-jz5kh\") pod \"isvc-canary-rollback-predictor-78cd59d4dc-ptxth\" (UID: \"a6d6b87d-3f49-4f74-9fc8-d9bcae4de021\") " pod="e2e-gw0-37d062d1dd/isvc-canary-rollback-predictor-78cd59d4dc-ptxth"
Oct 30 22:42:34.448264 ip-10-0-131-202 kubenswrapper[4412]: E1030 22:42:34.448233    4412 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bkhmz" containerName="extractor"
Oct 30 22:42:34.561101 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:42:34.561068    4412 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-37d062d1dd/isvc-canary-rollback-predictor-78cd59d4dc-ptxth"
Oct 30 22:42:34.984572 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:42:34.984548    4412 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw0-37d062d1dd/isvc-canary-rollback-predictor-78cd59d4dc-ptxth"]
Oct 30 22:42:34.986899 ip-10-0-131-202 kubenswrapper[4412]: W1030 22:42:34.986873    4412 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda6d6b87d_3f49_4f74_9fc8_d9bcae4de021.slice/crio-1279581f9909f50c56ea3ec069d5e68555999a2641e5653a86b0539fbc8d688e WatchSource:0}: Error finding container 1279581f9909f50c56ea3ec069d5e68555999a2641e5653a86b0539fbc8d688e: Status 404 returned error can't find the container with id 1279581f9909f50c56ea3ec069d5e68555999a2641e5653a86b0539fbc8d688e
Oct 30 22:42:35.604450 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:42:35.604414    4412 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-37d062d1dd/isvc-canary-rollback-predictor-78cd59d4dc-ptxth" event={"ID":"a6d6b87d-3f49-4f74-9fc8-d9bcae4de021","Type":"ContainerStarted","Data":"ea7d0fe417b61a47aef4a06c4381edfa040343ace84e4ee84c7c1513eb19b7f2"}
Oct 30 22:42:35.604699 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:42:35.604487    4412 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-37d062d1dd/isvc-canary-rollback-predictor-78cd59d4dc-ptxth" event={"ID":"a6d6b87d-3f49-4f74-9fc8-d9bcae4de021","Type":"ContainerStarted","Data":"1279581f9909f50c56ea3ec069d5e68555999a2641e5653a86b0539fbc8d688e"}
Oct 30 22:42:39.620292 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:42:39.620258    4412 generic.go:356] "Generic (PLEG): container finished" podID="a6d6b87d-3f49-4f74-9fc8-d9bcae4de021" containerID="ea7d0fe417b61a47aef4a06c4381edfa040343ace84e4ee84c7c1513eb19b7f2" exitCode=0
Oct 30 22:42:39.620780 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:42:39.620333    4412 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-37d062d1dd/isvc-canary-rollback-predictor-78cd59d4dc-ptxth" event={"ID":"a6d6b87d-3f49-4f74-9fc8-d9bcae4de021","Type":"ContainerDied","Data":"ea7d0fe417b61a47aef4a06c4381edfa040343ace84e4ee84c7c1513eb19b7f2"}
Oct 30 22:42:40.625905 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:42:40.625869    4412 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-37d062d1dd/isvc-canary-rollback-predictor-78cd59d4dc-ptxth" event={"ID":"a6d6b87d-3f49-4f74-9fc8-d9bcae4de021","Type":"ContainerStarted","Data":"35172a34c2fc5f61eaabf622a4ef499598330a0eba800e8141a855cf93e9cf54"}
Oct 30 22:42:40.626397 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:42:40.625953    4412 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-37d062d1dd/isvc-canary-rollback-predictor-78cd59d4dc-ptxth" event={"ID":"a6d6b87d-3f49-4f74-9fc8-d9bcae4de021","Type":"ContainerStarted","Data":"4d9320c9768035e66f55ba36dffa77060f4ca5c6a8546cc2e52cc3527f79b54a"}
Oct 30 22:42:40.626397 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:42:40.626232    4412 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw0-37d062d1dd/isvc-canary-rollback-predictor-78cd59d4dc-ptxth"
Oct 30 22:42:40.626397 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:42:40.626390    4412 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw0-37d062d1dd/isvc-canary-rollback-predictor-78cd59d4dc-ptxth"
Oct 30 22:42:40.627752 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:42:40.627729    4412 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-37d062d1dd/isvc-canary-rollback-predictor-78cd59d4dc-ptxth" podUID="a6d6b87d-3f49-4f74-9fc8-d9bcae4de021" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.31:8080: connect: connection refused"
Oct 30 22:42:40.795395 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:42:40.795343    4412 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-gw0-37d062d1dd/isvc-canary-rollback-predictor-78cd59d4dc-ptxth" podStartSLOduration=3.803203166 podStartE2EDuration="7.795329958s" totalImagesPullingTime="0s" totalInitContainerRuntime="3.992126792s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:42:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:42:40.79504021 +0000 UTC m=+2981.299712161" watchObservedRunningTime="2026-10-30 22:42:40.795329958 +0000 UTC m=+2981.300001891"
Oct 30 22:42:41.629493 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:42:41.629456    4412 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-37d062d1dd/isvc-canary-rollback-predictor-78cd59d4dc-ptxth" podUID="a6d6b87d-3f49-4f74-9fc8-d9bcae4de021" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.31:8080: connect: connection refused"
Oct 30 22:42:46.634604 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:42:46.634575    4412 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw0-37d062d1dd/isvc-canary-rollback-predictor-78cd59d4dc-ptxth"
Oct 30 22:42:46.635081 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:42:46.635052    4412 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-37d062d1dd/isvc-canary-rollback-predictor-78cd59d4dc-ptxth" podUID="a6d6b87d-3f49-4f74-9fc8-d9bcae4de021" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.31:8080: connect: connection refused"
Oct 30 22:42:56.635566 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:42:56.635530    4412 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-37d062d1dd/isvc-canary-rollback-predictor-78cd59d4dc-ptxth" podUID="a6d6b87d-3f49-4f74-9fc8-d9bcae4de021" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.31:8080: connect: connection refused"
Oct 30 22:43:04.451344 ip-10-0-131-202 kubenswrapper[4412]: E1030 22:43:04.451305    4412 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bkhmz" containerName="extractor"
Oct 30 22:43:06.635631 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:43:06.635592    4412 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-37d062d1dd/isvc-canary-rollback-predictor-78cd59d4dc-ptxth" podUID="a6d6b87d-3f49-4f74-9fc8-d9bcae4de021" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.31:8080: connect: connection refused"
Oct 30 22:43:16.635595 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:43:16.635511    4412 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-37d062d1dd/isvc-canary-rollback-predictor-78cd59d4dc-ptxth" podUID="a6d6b87d-3f49-4f74-9fc8-d9bcae4de021" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.31:8080: connect: connection refused"
Oct 30 22:43:26.635233 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:43:26.635197    4412 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-37d062d1dd/isvc-canary-rollback-predictor-78cd59d4dc-ptxth" podUID="a6d6b87d-3f49-4f74-9fc8-d9bcae4de021" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.31:8080: connect: connection refused"
Oct 30 22:43:34.448622 ip-10-0-131-202 kubenswrapper[4412]: E1030 22:43:34.448583    4412 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bkhmz" containerName="extractor"
Oct 30 22:43:36.635951 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:43:36.635910    4412 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-37d062d1dd/isvc-canary-rollback-predictor-78cd59d4dc-ptxth" podUID="a6d6b87d-3f49-4f74-9fc8-d9bcae4de021" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.31:8080: connect: connection refused"
Oct 30 22:43:46.635847 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:43:46.635808    4412 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-37d062d1dd/isvc-canary-rollback-predictor-78cd59d4dc-ptxth" podUID="a6d6b87d-3f49-4f74-9fc8-d9bcae4de021" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.31:8080: connect: connection refused"
Oct 30 22:43:56.636303 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:43:56.636276    4412 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw0-37d062d1dd/isvc-canary-rollback-predictor-78cd59d4dc-ptxth"
Oct 30 22:44:04.448495 ip-10-0-131-202 kubenswrapper[4412]: E1030 22:44:04.448455    4412 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bkhmz" containerName="extractor"
Oct 30 22:44:34.448220 ip-10-0-131-202 kubenswrapper[4412]: E1030 22:44:34.448185    4412 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bkhmz" containerName="extractor"
Oct 30 22:45:04.448749 ip-10-0-131-202 kubenswrapper[4412]: E1030 22:45:04.448665    4412 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bkhmz" containerName="extractor"
Oct 30 22:45:34.450237 ip-10-0-131-202 kubenswrapper[4412]: E1030 22:45:34.450200    4412 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bkhmz" containerName="extractor"
Oct 30 22:45:43.569798 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:45:43.569765    4412 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw0-37d062d1dd/isvc-canary-rollback-predictor-78cd59d4dc-ptxth"]
Oct 30 22:45:43.570391 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:45:43.570057    4412 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw0-37d062d1dd/isvc-canary-rollback-predictor-78cd59d4dc-ptxth" podUID="a6d6b87d-3f49-4f74-9fc8-d9bcae4de021" containerName="kserve-container" containerID="cri-o://4d9320c9768035e66f55ba36dffa77060f4ca5c6a8546cc2e52cc3527f79b54a" gracePeriod=30
Oct 30 22:45:43.570391 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:45:43.570145    4412 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw0-37d062d1dd/isvc-canary-rollback-predictor-78cd59d4dc-ptxth" podUID="a6d6b87d-3f49-4f74-9fc8-d9bcae4de021" containerName="kube-rbac-proxy" containerID="cri-o://35172a34c2fc5f61eaabf622a4ef499598330a0eba800e8141a855cf93e9cf54" gracePeriod=30
Oct 30 22:45:44.299783 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:45:44.299751    4412 generic.go:356] "Generic (PLEG): container finished" podID="a6d6b87d-3f49-4f74-9fc8-d9bcae4de021" containerID="35172a34c2fc5f61eaabf622a4ef499598330a0eba800e8141a855cf93e9cf54" exitCode=0
Oct 30 22:45:44.299948 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:45:44.299803    4412 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-37d062d1dd/isvc-canary-rollback-predictor-78cd59d4dc-ptxth" event={"ID":"a6d6b87d-3f49-4f74-9fc8-d9bcae4de021","Type":"ContainerDied","Data":"35172a34c2fc5f61eaabf622a4ef499598330a0eba800e8141a855cf93e9cf54"}
Oct 30 22:45:46.630280 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:45:46.630239    4412 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-37d062d1dd/isvc-canary-rollback-predictor-78cd59d4dc-ptxth" podUID="a6d6b87d-3f49-4f74-9fc8-d9bcae4de021" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.31:8643/healthz\": dial tcp 10.133.0.31:8643: connect: connection refused"
Oct 30 22:45:46.635593 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:45:46.635574    4412 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-37d062d1dd/isvc-canary-rollback-predictor-78cd59d4dc-ptxth" podUID="a6d6b87d-3f49-4f74-9fc8-d9bcae4de021" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.31:8080: connect: connection refused"
Oct 30 22:45:48.212384 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:45:48.212357    4412 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-37d062d1dd/isvc-canary-rollback-predictor-78cd59d4dc-ptxth"
Oct 30 22:45:48.228452 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:45:48.228431    4412 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a6d6b87d-3f49-4f74-9fc8-d9bcae4de021-kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/a6d6b87d-3f49-4f74-9fc8-d9bcae4de021-kube-rbac-proxy-sar-config\") pod \"a6d6b87d-3f49-4f74-9fc8-d9bcae4de021\" (UID: \"a6d6b87d-3f49-4f74-9fc8-d9bcae4de021\") "
Oct 30 22:45:48.228542 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:45:48.228477    4412 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a6d6b87d-3f49-4f74-9fc8-d9bcae4de021-kube-api-access-jz5kh\" (UniqueName: \"kubernetes.io/projected/a6d6b87d-3f49-4f74-9fc8-d9bcae4de021-kube-api-access-jz5kh\") pod \"a6d6b87d-3f49-4f74-9fc8-d9bcae4de021\" (UID: \"a6d6b87d-3f49-4f74-9fc8-d9bcae4de021\") "
Oct 30 22:45:48.228542 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:45:48.228523    4412 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a6d6b87d-3f49-4f74-9fc8-d9bcae4de021-kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/a6d6b87d-3f49-4f74-9fc8-d9bcae4de021-kserve-provision-location\") pod \"a6d6b87d-3f49-4f74-9fc8-d9bcae4de021\" (UID: \"a6d6b87d-3f49-4f74-9fc8-d9bcae4de021\") "
Oct 30 22:45:48.228629 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:45:48.228548    4412 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a6d6b87d-3f49-4f74-9fc8-d9bcae4de021-proxy-tls\" (UniqueName: \"kubernetes.io/secret/a6d6b87d-3f49-4f74-9fc8-d9bcae4de021-proxy-tls\") pod \"a6d6b87d-3f49-4f74-9fc8-d9bcae4de021\" (UID: \"a6d6b87d-3f49-4f74-9fc8-d9bcae4de021\") "
Oct 30 22:45:48.228629 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:45:48.228599    4412 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a6d6b87d-3f49-4f74-9fc8-d9bcae4de021-cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/a6d6b87d-3f49-4f74-9fc8-d9bcae4de021-cabundle-cert\") pod \"a6d6b87d-3f49-4f74-9fc8-d9bcae4de021\" (UID: \"a6d6b87d-3f49-4f74-9fc8-d9bcae4de021\") "
Oct 30 22:45:48.228817 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:45:48.228791    4412 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a6d6b87d-3f49-4f74-9fc8-d9bcae4de021-kube-rbac-proxy-sar-config" pod "a6d6b87d-3f49-4f74-9fc8-d9bcae4de021" (UID: "a6d6b87d-3f49-4f74-9fc8-d9bcae4de021"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:45:48.228887 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:45:48.228854    4412 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a6d6b87d-3f49-4f74-9fc8-d9bcae4de021-kserve-provision-location" pod "a6d6b87d-3f49-4f74-9fc8-d9bcae4de021" (UID: "a6d6b87d-3f49-4f74-9fc8-d9bcae4de021"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:45:48.229042 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:45:48.229022    4412 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a6d6b87d-3f49-4f74-9fc8-d9bcae4de021-cabundle-cert" pod "a6d6b87d-3f49-4f74-9fc8-d9bcae4de021" (UID: "a6d6b87d-3f49-4f74-9fc8-d9bcae4de021"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:45:48.230741 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:45:48.230724    4412 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a6d6b87d-3f49-4f74-9fc8-d9bcae4de021-proxy-tls" pod "a6d6b87d-3f49-4f74-9fc8-d9bcae4de021" (UID: "a6d6b87d-3f49-4f74-9fc8-d9bcae4de021"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:45:48.230850 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:45:48.230828    4412 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a6d6b87d-3f49-4f74-9fc8-d9bcae4de021-kube-api-access-jz5kh" pod "a6d6b87d-3f49-4f74-9fc8-d9bcae4de021" (UID: "a6d6b87d-3f49-4f74-9fc8-d9bcae4de021"). InnerVolumeSpecName "kube-api-access-jz5kh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:45:48.317586 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:45:48.317508    4412 generic.go:356] "Generic (PLEG): container finished" podID="a6d6b87d-3f49-4f74-9fc8-d9bcae4de021" containerID="4d9320c9768035e66f55ba36dffa77060f4ca5c6a8546cc2e52cc3527f79b54a" exitCode=0
Oct 30 22:45:48.317586 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:45:48.317567    4412 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-37d062d1dd/isvc-canary-rollback-predictor-78cd59d4dc-ptxth"
Oct 30 22:45:48.317796 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:45:48.317557    4412 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-37d062d1dd/isvc-canary-rollback-predictor-78cd59d4dc-ptxth" event={"ID":"a6d6b87d-3f49-4f74-9fc8-d9bcae4de021","Type":"ContainerDied","Data":"4d9320c9768035e66f55ba36dffa77060f4ca5c6a8546cc2e52cc3527f79b54a"}
Oct 30 22:45:48.317796 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:45:48.317703    4412 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-37d062d1dd/isvc-canary-rollback-predictor-78cd59d4dc-ptxth" event={"ID":"a6d6b87d-3f49-4f74-9fc8-d9bcae4de021","Type":"ContainerDied","Data":"1279581f9909f50c56ea3ec069d5e68555999a2641e5653a86b0539fbc8d688e"}
Oct 30 22:45:48.317796 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:45:48.317741    4412 scope.go:122] "RemoveContainer" containerID="35172a34c2fc5f61eaabf622a4ef499598330a0eba800e8141a855cf93e9cf54"
Oct 30 22:45:48.326649 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:45:48.326633    4412 scope.go:122] "RemoveContainer" containerID="4d9320c9768035e66f55ba36dffa77060f4ca5c6a8546cc2e52cc3527f79b54a"
Oct 30 22:45:48.329980 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:45:48.329957    4412 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/a6d6b87d-3f49-4f74-9fc8-d9bcae4de021-kserve-provision-location\") on node \"ip-10-0-131-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:45:48.330058 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:45:48.329987    4412 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/a6d6b87d-3f49-4f74-9fc8-d9bcae4de021-proxy-tls\") on node \"ip-10-0-131-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:45:48.330058 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:45:48.330003    4412 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/a6d6b87d-3f49-4f74-9fc8-d9bcae4de021-cabundle-cert\") on node \"ip-10-0-131-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:45:48.330058 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:45:48.330018    4412 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/a6d6b87d-3f49-4f74-9fc8-d9bcae4de021-kube-rbac-proxy-sar-config\") on node \"ip-10-0-131-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:45:48.330058 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:45:48.330032    4412 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jz5kh\" (UniqueName: \"kubernetes.io/projected/a6d6b87d-3f49-4f74-9fc8-d9bcae4de021-kube-api-access-jz5kh\") on node \"ip-10-0-131-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:45:48.334012 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:45:48.333995    4412 scope.go:122] "RemoveContainer" containerID="ea7d0fe417b61a47aef4a06c4381edfa040343ace84e4ee84c7c1513eb19b7f2"
Oct 30 22:45:48.340827 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:45:48.340807    4412 scope.go:122] "RemoveContainer" containerID="35172a34c2fc5f61eaabf622a4ef499598330a0eba800e8141a855cf93e9cf54"
Oct 30 22:45:48.341096 ip-10-0-131-202 kubenswrapper[4412]: E1030 22:45:48.341080    4412 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"35172a34c2fc5f61eaabf622a4ef499598330a0eba800e8141a855cf93e9cf54\": container with ID starting with 35172a34c2fc5f61eaabf622a4ef499598330a0eba800e8141a855cf93e9cf54 not found: ID does not exist" containerID="35172a34c2fc5f61eaabf622a4ef499598330a0eba800e8141a855cf93e9cf54"
Oct 30 22:45:48.341215 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:45:48.341105    4412 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"35172a34c2fc5f61eaabf622a4ef499598330a0eba800e8141a855cf93e9cf54"} err="failed to get container status \"35172a34c2fc5f61eaabf622a4ef499598330a0eba800e8141a855cf93e9cf54\": rpc error: code = NotFound desc = could not find container \"35172a34c2fc5f61eaabf622a4ef499598330a0eba800e8141a855cf93e9cf54\": container with ID starting with 35172a34c2fc5f61eaabf622a4ef499598330a0eba800e8141a855cf93e9cf54 not found: ID does not exist"
Oct 30 22:45:48.341263 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:45:48.341218    4412 scope.go:122] "RemoveContainer" containerID="4d9320c9768035e66f55ba36dffa77060f4ca5c6a8546cc2e52cc3527f79b54a"
Oct 30 22:45:48.341470 ip-10-0-131-202 kubenswrapper[4412]: E1030 22:45:48.341452    4412 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4d9320c9768035e66f55ba36dffa77060f4ca5c6a8546cc2e52cc3527f79b54a\": container with ID starting with 4d9320c9768035e66f55ba36dffa77060f4ca5c6a8546cc2e52cc3527f79b54a not found: ID does not exist" containerID="4d9320c9768035e66f55ba36dffa77060f4ca5c6a8546cc2e52cc3527f79b54a"
Oct 30 22:45:48.341516 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:45:48.341474    4412 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4d9320c9768035e66f55ba36dffa77060f4ca5c6a8546cc2e52cc3527f79b54a"} err="failed to get container status \"4d9320c9768035e66f55ba36dffa77060f4ca5c6a8546cc2e52cc3527f79b54a\": rpc error: code = NotFound desc = could not find container \"4d9320c9768035e66f55ba36dffa77060f4ca5c6a8546cc2e52cc3527f79b54a\": container with ID starting with 4d9320c9768035e66f55ba36dffa77060f4ca5c6a8546cc2e52cc3527f79b54a not found: ID does not exist"
Oct 30 22:45:48.341516 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:45:48.341489    4412 scope.go:122] "RemoveContainer" containerID="ea7d0fe417b61a47aef4a06c4381edfa040343ace84e4ee84c7c1513eb19b7f2"
Oct 30 22:45:48.341729 ip-10-0-131-202 kubenswrapper[4412]: E1030 22:45:48.341714    4412 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ea7d0fe417b61a47aef4a06c4381edfa040343ace84e4ee84c7c1513eb19b7f2\": container with ID starting with ea7d0fe417b61a47aef4a06c4381edfa040343ace84e4ee84c7c1513eb19b7f2 not found: ID does not exist" containerID="ea7d0fe417b61a47aef4a06c4381edfa040343ace84e4ee84c7c1513eb19b7f2"
Oct 30 22:45:48.341772 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:45:48.341733    4412 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ea7d0fe417b61a47aef4a06c4381edfa040343ace84e4ee84c7c1513eb19b7f2"} err="failed to get container status \"ea7d0fe417b61a47aef4a06c4381edfa040343ace84e4ee84c7c1513eb19b7f2\": rpc error: code = NotFound desc = could not find container \"ea7d0fe417b61a47aef4a06c4381edfa040343ace84e4ee84c7c1513eb19b7f2\": container with ID starting with ea7d0fe417b61a47aef4a06c4381edfa040343ace84e4ee84c7c1513eb19b7f2 not found: ID does not exist"
Oct 30 22:45:48.495447 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:45:48.495420    4412 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw0-37d062d1dd/isvc-canary-rollback-predictor-78cd59d4dc-ptxth"]
Oct 30 22:45:48.503364 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:45:48.503340    4412 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-gw0-37d062d1dd/isvc-canary-rollback-predictor-78cd59d4dc-ptxth"]
Oct 30 22:45:49.626631 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:45:49.626598    4412 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a6d6b87d-3f49-4f74-9fc8-d9bcae4de021" path="/var/lib/kubelet/pods/a6d6b87d-3f49-4f74-9fc8-d9bcae4de021/volumes"
Oct 30 22:45:53.929132 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:45:53.929082    4412 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-gw0-37d062d1dd/isvc-canary-stop-predictor-748fd79575-vr6cb"]
Oct 30 22:45:53.929504 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:45:53.929386    4412 cpu_manager.go:414] "containerMap: removing container" podUID="a6d6b87d-3f49-4f74-9fc8-d9bcae4de021" containerName="kube-rbac-proxy"
Oct 30 22:45:53.929504 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:45:53.929399    4412 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a6d6b87d-3f49-4f74-9fc8-d9bcae4de021" containerName="kube-rbac-proxy"
Oct 30 22:45:53.929504 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:45:53.929424    4412 cpu_manager.go:414] "containerMap: removing container" podUID="a6d6b87d-3f49-4f74-9fc8-d9bcae4de021" containerName="storage-initializer"
Oct 30 22:45:53.929504 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:45:53.929429    4412 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a6d6b87d-3f49-4f74-9fc8-d9bcae4de021" containerName="storage-initializer"
Oct 30 22:45:53.929504 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:45:53.929436    4412 cpu_manager.go:414] "containerMap: removing container" podUID="a6d6b87d-3f49-4f74-9fc8-d9bcae4de021" containerName="kserve-container"
Oct 30 22:45:53.929504 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:45:53.929441    4412 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a6d6b87d-3f49-4f74-9fc8-d9bcae4de021" containerName="kserve-container"
Oct 30 22:45:53.929504 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:45:53.929491    4412 memory_manager.go:365] "RemoveStaleState removing state" podUID="a6d6b87d-3f49-4f74-9fc8-d9bcae4de021" containerName="kserve-container"
Oct 30 22:45:53.929504 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:45:53.929502    4412 memory_manager.go:365] "RemoveStaleState removing state" podUID="a6d6b87d-3f49-4f74-9fc8-d9bcae4de021" containerName="kube-rbac-proxy"
Oct 30 22:45:53.934066 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:45:53.934050    4412 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-37d062d1dd/isvc-canary-stop-predictor-748fd79575-vr6cb"
Oct 30 22:45:53.973450 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:45:53.973420    4412 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/f02602f2-1009-47b0-a05a-e69c2a9ebf6f-proxy-tls\") pod \"isvc-canary-stop-predictor-748fd79575-vr6cb\" (UID: \"f02602f2-1009-47b0-a05a-e69c2a9ebf6f\") " pod="e2e-gw0-37d062d1dd/isvc-canary-stop-predictor-748fd79575-vr6cb"
Oct 30 22:45:53.973575 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:45:53.973454    4412 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xqnwd\" (UniqueName: \"kubernetes.io/projected/f02602f2-1009-47b0-a05a-e69c2a9ebf6f-kube-api-access-xqnwd\") pod \"isvc-canary-stop-predictor-748fd79575-vr6cb\" (UID: \"f02602f2-1009-47b0-a05a-e69c2a9ebf6f\") " pod="e2e-gw0-37d062d1dd/isvc-canary-stop-predictor-748fd79575-vr6cb"
Oct 30 22:45:53.973575 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:45:53.973494    4412 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/f02602f2-1009-47b0-a05a-e69c2a9ebf6f-kube-rbac-proxy-sar-config\") pod \"isvc-canary-stop-predictor-748fd79575-vr6cb\" (UID: \"f02602f2-1009-47b0-a05a-e69c2a9ebf6f\") " pod="e2e-gw0-37d062d1dd/isvc-canary-stop-predictor-748fd79575-vr6cb"
Oct 30 22:45:53.973653 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:45:53.973606    4412 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/f02602f2-1009-47b0-a05a-e69c2a9ebf6f-kserve-provision-location\") pod \"isvc-canary-stop-predictor-748fd79575-vr6cb\" (UID: \"f02602f2-1009-47b0-a05a-e69c2a9ebf6f\") " pod="e2e-gw0-37d062d1dd/isvc-canary-stop-predictor-748fd79575-vr6cb"
Oct 30 22:45:53.973653 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:45:53.973634    4412 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/f02602f2-1009-47b0-a05a-e69c2a9ebf6f-cabundle-cert\") pod \"isvc-canary-stop-predictor-748fd79575-vr6cb\" (UID: \"f02602f2-1009-47b0-a05a-e69c2a9ebf6f\") " pod="e2e-gw0-37d062d1dd/isvc-canary-stop-predictor-748fd79575-vr6cb"
Oct 30 22:45:54.037636 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:45:54.037605    4412 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw0-37d062d1dd\"/\"isvc-canary-stop-kube-rbac-proxy-sar-config\""
Oct 30 22:45:54.037636 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:45:54.037623    4412 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw0-37d062d1dd\"/\"kube-root-ca.crt\""
Oct 30 22:45:54.037855 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:45:54.037673    4412 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw0-37d062d1dd\"/\"isvc-canary-stop-predictor-serving-cert\""
Oct 30 22:45:54.037855 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:45:54.037689    4412 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw0-37d062d1dd\"/\"odh-kserve-custom-ca-bundle\""
Oct 30 22:45:54.037855 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:45:54.037716    4412 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw0-37d062d1dd\"/\"openshift-service-ca.crt\""
Oct 30 22:45:54.037855 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:45:54.037740    4412 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw0-37d062d1dd\"/\"default-dockercfg-4v4s8\""
Oct 30 22:45:54.037855 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:45:54.037770    4412 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw0-37d062d1dd\"/\"seaweedfs-s3-creds\""
Oct 30 22:45:54.039068 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:45:54.039045    4412 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw0-37d062d1dd/isvc-canary-stop-predictor-748fd79575-vr6cb"]
Oct 30 22:45:54.073961 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:45:54.073939    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/f02602f2-1009-47b0-a05a-e69c2a9ebf6f-kserve-provision-location\") pod \"isvc-canary-stop-predictor-748fd79575-vr6cb\" (UID: \"f02602f2-1009-47b0-a05a-e69c2a9ebf6f\") " pod="e2e-gw0-37d062d1dd/isvc-canary-stop-predictor-748fd79575-vr6cb"
Oct 30 22:45:54.074046 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:45:54.073963    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/f02602f2-1009-47b0-a05a-e69c2a9ebf6f-cabundle-cert\") pod \"isvc-canary-stop-predictor-748fd79575-vr6cb\" (UID: \"f02602f2-1009-47b0-a05a-e69c2a9ebf6f\") " pod="e2e-gw0-37d062d1dd/isvc-canary-stop-predictor-748fd79575-vr6cb"
Oct 30 22:45:54.074046 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:45:54.073993    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/f02602f2-1009-47b0-a05a-e69c2a9ebf6f-proxy-tls\") pod \"isvc-canary-stop-predictor-748fd79575-vr6cb\" (UID: \"f02602f2-1009-47b0-a05a-e69c2a9ebf6f\") " pod="e2e-gw0-37d062d1dd/isvc-canary-stop-predictor-748fd79575-vr6cb"
Oct 30 22:45:54.074046 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:45:54.074011    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xqnwd\" (UniqueName: \"kubernetes.io/projected/f02602f2-1009-47b0-a05a-e69c2a9ebf6f-kube-api-access-xqnwd\") pod \"isvc-canary-stop-predictor-748fd79575-vr6cb\" (UID: \"f02602f2-1009-47b0-a05a-e69c2a9ebf6f\") " pod="e2e-gw0-37d062d1dd/isvc-canary-stop-predictor-748fd79575-vr6cb"
Oct 30 22:45:54.074237 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:45:54.074051    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/f02602f2-1009-47b0-a05a-e69c2a9ebf6f-kube-rbac-proxy-sar-config\") pod \"isvc-canary-stop-predictor-748fd79575-vr6cb\" (UID: \"f02602f2-1009-47b0-a05a-e69c2a9ebf6f\") " pod="e2e-gw0-37d062d1dd/isvc-canary-stop-predictor-748fd79575-vr6cb"
Oct 30 22:45:54.074417 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:45:54.074394    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/f02602f2-1009-47b0-a05a-e69c2a9ebf6f-kserve-provision-location\") pod \"isvc-canary-stop-predictor-748fd79575-vr6cb\" (UID: \"f02602f2-1009-47b0-a05a-e69c2a9ebf6f\") " pod="e2e-gw0-37d062d1dd/isvc-canary-stop-predictor-748fd79575-vr6cb"
Oct 30 22:45:54.074657 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:45:54.074627    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/f02602f2-1009-47b0-a05a-e69c2a9ebf6f-cabundle-cert\") pod \"isvc-canary-stop-predictor-748fd79575-vr6cb\" (UID: \"f02602f2-1009-47b0-a05a-e69c2a9ebf6f\") " pod="e2e-gw0-37d062d1dd/isvc-canary-stop-predictor-748fd79575-vr6cb"
Oct 30 22:45:54.074754 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:45:54.074696    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/f02602f2-1009-47b0-a05a-e69c2a9ebf6f-kube-rbac-proxy-sar-config\") pod \"isvc-canary-stop-predictor-748fd79575-vr6cb\" (UID: \"f02602f2-1009-47b0-a05a-e69c2a9ebf6f\") " pod="e2e-gw0-37d062d1dd/isvc-canary-stop-predictor-748fd79575-vr6cb"
Oct 30 22:45:54.076717 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:45:54.076696    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/f02602f2-1009-47b0-a05a-e69c2a9ebf6f-proxy-tls\") pod \"isvc-canary-stop-predictor-748fd79575-vr6cb\" (UID: \"f02602f2-1009-47b0-a05a-e69c2a9ebf6f\") " pod="e2e-gw0-37d062d1dd/isvc-canary-stop-predictor-748fd79575-vr6cb"
Oct 30 22:45:54.145642 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:45:54.145611    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xqnwd\" (UniqueName: \"kubernetes.io/projected/f02602f2-1009-47b0-a05a-e69c2a9ebf6f-kube-api-access-xqnwd\") pod \"isvc-canary-stop-predictor-748fd79575-vr6cb\" (UID: \"f02602f2-1009-47b0-a05a-e69c2a9ebf6f\") " pod="e2e-gw0-37d062d1dd/isvc-canary-stop-predictor-748fd79575-vr6cb"
Oct 30 22:45:54.250796 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:45:54.250766    4412 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-37d062d1dd/isvc-canary-stop-predictor-748fd79575-vr6cb"
Oct 30 22:45:54.673353 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:45:54.673277    4412 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw0-37d062d1dd/isvc-canary-stop-predictor-748fd79575-vr6cb"]
Oct 30 22:45:54.676648 ip-10-0-131-202 kubenswrapper[4412]: W1030 22:45:54.676623    4412 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf02602f2_1009_47b0_a05a_e69c2a9ebf6f.slice/crio-d8ecc1dbd12ddf37dc85edf3da898933ac4b19c2841ed65449134681651bca2d WatchSource:0}: Error finding container d8ecc1dbd12ddf37dc85edf3da898933ac4b19c2841ed65449134681651bca2d: Status 404 returned error can't find the container with id d8ecc1dbd12ddf37dc85edf3da898933ac4b19c2841ed65449134681651bca2d
Oct 30 22:45:55.345227 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:45:55.345190    4412 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-37d062d1dd/isvc-canary-stop-predictor-748fd79575-vr6cb" event={"ID":"f02602f2-1009-47b0-a05a-e69c2a9ebf6f","Type":"ContainerStarted","Data":"48355f431b12a16b84f0a3aef909b2353c2e51b4f1e2a66310573aef2261372f"}
Oct 30 22:45:55.345692 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:45:55.345246    4412 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-37d062d1dd/isvc-canary-stop-predictor-748fd79575-vr6cb" event={"ID":"f02602f2-1009-47b0-a05a-e69c2a9ebf6f","Type":"ContainerStarted","Data":"d8ecc1dbd12ddf37dc85edf3da898933ac4b19c2841ed65449134681651bca2d"}
Oct 30 22:45:59.361303 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:45:59.361266    4412 generic.go:356] "Generic (PLEG): container finished" podID="f02602f2-1009-47b0-a05a-e69c2a9ebf6f" containerID="48355f431b12a16b84f0a3aef909b2353c2e51b4f1e2a66310573aef2261372f" exitCode=0
Oct 30 22:45:59.361655 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:45:59.361315    4412 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-37d062d1dd/isvc-canary-stop-predictor-748fd79575-vr6cb" event={"ID":"f02602f2-1009-47b0-a05a-e69c2a9ebf6f","Type":"ContainerDied","Data":"48355f431b12a16b84f0a3aef909b2353c2e51b4f1e2a66310573aef2261372f"}
Oct 30 22:46:00.367333 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:46:00.367257    4412 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-37d062d1dd/isvc-canary-stop-predictor-748fd79575-vr6cb" event={"ID":"f02602f2-1009-47b0-a05a-e69c2a9ebf6f","Type":"ContainerStarted","Data":"e267e8357ff576e471bc4b993510394dcc01aee681f812ef786a7768a8c9ff52"}
Oct 30 22:46:00.367333 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:46:00.367320    4412 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-37d062d1dd/isvc-canary-stop-predictor-748fd79575-vr6cb" event={"ID":"f02602f2-1009-47b0-a05a-e69c2a9ebf6f","Type":"ContainerStarted","Data":"24bb7c75fb96faebf5e9de8d9a5323607ae1ee20061ca7f96de19b3b3bd1c882"}
Oct 30 22:46:00.367764 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:46:00.367601    4412 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw0-37d062d1dd/isvc-canary-stop-predictor-748fd79575-vr6cb"
Oct 30 22:46:00.367764 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:46:00.367647    4412 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw0-37d062d1dd/isvc-canary-stop-predictor-748fd79575-vr6cb"
Oct 30 22:46:00.369024 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:46:00.368997    4412 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-37d062d1dd/isvc-canary-stop-predictor-748fd79575-vr6cb" podUID="f02602f2-1009-47b0-a05a-e69c2a9ebf6f" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.32:8080: connect: connection refused"
Oct 30 22:46:00.490960 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:46:00.490903    4412 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-gw0-37d062d1dd/isvc-canary-stop-predictor-748fd79575-vr6cb" podStartSLOduration=3.297184753 podStartE2EDuration="7.490890768s" totalImagesPullingTime="0s" totalInitContainerRuntime="4.193706015s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:45:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:46:00.489560146 +0000 UTC m=+3180.994232097" watchObservedRunningTime="2026-10-30 22:46:00.490890768 +0000 UTC m=+3180.995562720"
Oct 30 22:46:01.371348 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:46:01.371312    4412 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-37d062d1dd/isvc-canary-stop-predictor-748fd79575-vr6cb" podUID="f02602f2-1009-47b0-a05a-e69c2a9ebf6f" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.32:8080: connect: connection refused"
Oct 30 22:46:04.447795 ip-10-0-131-202 kubenswrapper[4412]: E1030 22:46:04.447755    4412 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bkhmz" containerName="extractor"
Oct 30 22:46:06.375888 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:46:06.375856    4412 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw0-37d062d1dd/isvc-canary-stop-predictor-748fd79575-vr6cb"
Oct 30 22:46:06.376426 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:46:06.376320    4412 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-37d062d1dd/isvc-canary-stop-predictor-748fd79575-vr6cb" podUID="f02602f2-1009-47b0-a05a-e69c2a9ebf6f" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.32:8080: connect: connection refused"
Oct 30 22:46:16.376324 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:46:16.376285    4412 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-37d062d1dd/isvc-canary-stop-predictor-748fd79575-vr6cb" podUID="f02602f2-1009-47b0-a05a-e69c2a9ebf6f" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.32:8080: connect: connection refused"
Oct 30 22:46:26.377263 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:46:26.377226    4412 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-37d062d1dd/isvc-canary-stop-predictor-748fd79575-vr6cb" podUID="f02602f2-1009-47b0-a05a-e69c2a9ebf6f" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.32:8080: connect: connection refused"
Oct 30 22:46:34.447501 ip-10-0-131-202 kubenswrapper[4412]: E1030 22:46:34.447464    4412 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bkhmz" containerName="extractor"
Oct 30 22:46:36.377087 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:46:36.377048    4412 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-37d062d1dd/isvc-canary-stop-predictor-748fd79575-vr6cb" podUID="f02602f2-1009-47b0-a05a-e69c2a9ebf6f" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.32:8080: connect: connection refused"
Oct 30 22:46:46.376843 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:46:46.376807    4412 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-37d062d1dd/isvc-canary-stop-predictor-748fd79575-vr6cb" podUID="f02602f2-1009-47b0-a05a-e69c2a9ebf6f" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.32:8080: connect: connection refused"
Oct 30 22:46:56.377131 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:46:56.377085    4412 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-37d062d1dd/isvc-canary-stop-predictor-748fd79575-vr6cb" podUID="f02602f2-1009-47b0-a05a-e69c2a9ebf6f" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.32:8080: connect: connection refused"
Oct 30 22:47:04.448722 ip-10-0-131-202 kubenswrapper[4412]: E1030 22:47:04.448679    4412 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bkhmz" containerName="extractor"
Oct 30 22:47:06.377172 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:47:06.377110    4412 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-37d062d1dd/isvc-canary-stop-predictor-748fd79575-vr6cb" podUID="f02602f2-1009-47b0-a05a-e69c2a9ebf6f" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.32:8080: connect: connection refused"
Oct 30 22:47:16.376974 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:47:16.376941    4412 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw0-37d062d1dd/isvc-canary-stop-predictor-748fd79575-vr6cb"
Oct 30 22:47:23.703971 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:47:23.703932    4412 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw0-37d062d1dd/isvc-canary-stop-predictor-748fd79575-vr6cb"]
Oct 30 22:47:23.704414 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:47:23.704163    4412 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw0-37d062d1dd/isvc-canary-stop-predictor-748fd79575-vr6cb" podUID="f02602f2-1009-47b0-a05a-e69c2a9ebf6f" containerName="kserve-container" containerID="cri-o://24bb7c75fb96faebf5e9de8d9a5323607ae1ee20061ca7f96de19b3b3bd1c882" gracePeriod=30
Oct 30 22:47:23.704414 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:47:23.704238    4412 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw0-37d062d1dd/isvc-canary-stop-predictor-748fd79575-vr6cb" podUID="f02602f2-1009-47b0-a05a-e69c2a9ebf6f" containerName="kube-rbac-proxy" containerID="cri-o://e267e8357ff576e471bc4b993510394dcc01aee681f812ef786a7768a8c9ff52" gracePeriod=30
Oct 30 22:47:24.667883 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:47:24.667844    4412 generic.go:356] "Generic (PLEG): container finished" podID="f02602f2-1009-47b0-a05a-e69c2a9ebf6f" containerID="e267e8357ff576e471bc4b993510394dcc01aee681f812ef786a7768a8c9ff52" exitCode=0
Oct 30 22:47:24.668057 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:47:24.667913    4412 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-37d062d1dd/isvc-canary-stop-predictor-748fd79575-vr6cb" event={"ID":"f02602f2-1009-47b0-a05a-e69c2a9ebf6f","Type":"ContainerDied","Data":"e267e8357ff576e471bc4b993510394dcc01aee681f812ef786a7768a8c9ff52"}
Oct 30 22:47:26.372432 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:47:26.372395    4412 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-37d062d1dd/isvc-canary-stop-predictor-748fd79575-vr6cb" podUID="f02602f2-1009-47b0-a05a-e69c2a9ebf6f" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.32:8643/healthz\": dial tcp 10.133.0.32:8643: connect: connection refused"
Oct 30 22:47:26.376714 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:47:26.376693    4412 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-37d062d1dd/isvc-canary-stop-predictor-748fd79575-vr6cb" podUID="f02602f2-1009-47b0-a05a-e69c2a9ebf6f" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.32:8080: connect: connection refused"
Oct 30 22:47:28.248768 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:47:28.248745    4412 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-37d062d1dd/isvc-canary-stop-predictor-748fd79575-vr6cb"
Oct 30 22:47:28.288928 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:47:28.288893    4412 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f02602f2-1009-47b0-a05a-e69c2a9ebf6f-cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/f02602f2-1009-47b0-a05a-e69c2a9ebf6f-cabundle-cert\") pod \"f02602f2-1009-47b0-a05a-e69c2a9ebf6f\" (UID: \"f02602f2-1009-47b0-a05a-e69c2a9ebf6f\") "
Oct 30 22:47:28.289035 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:47:28.288960    4412 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f02602f2-1009-47b0-a05a-e69c2a9ebf6f-kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/f02602f2-1009-47b0-a05a-e69c2a9ebf6f-kserve-provision-location\") pod \"f02602f2-1009-47b0-a05a-e69c2a9ebf6f\" (UID: \"f02602f2-1009-47b0-a05a-e69c2a9ebf6f\") "
Oct 30 22:47:28.289035 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:47:28.288990    4412 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f02602f2-1009-47b0-a05a-e69c2a9ebf6f-kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/f02602f2-1009-47b0-a05a-e69c2a9ebf6f-kube-rbac-proxy-sar-config\") pod \"f02602f2-1009-47b0-a05a-e69c2a9ebf6f\" (UID: \"f02602f2-1009-47b0-a05a-e69c2a9ebf6f\") "
Oct 30 22:47:28.289105 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:47:28.289048    4412 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/f02602f2-1009-47b0-a05a-e69c2a9ebf6f-proxy-tls\" (UniqueName: \"kubernetes.io/secret/f02602f2-1009-47b0-a05a-e69c2a9ebf6f-proxy-tls\") pod \"f02602f2-1009-47b0-a05a-e69c2a9ebf6f\" (UID: \"f02602f2-1009-47b0-a05a-e69c2a9ebf6f\") "
Oct 30 22:47:28.289105 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:47:28.289070    4412 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f02602f2-1009-47b0-a05a-e69c2a9ebf6f-kube-api-access-xqnwd\" (UniqueName: \"kubernetes.io/projected/f02602f2-1009-47b0-a05a-e69c2a9ebf6f-kube-api-access-xqnwd\") pod \"f02602f2-1009-47b0-a05a-e69c2a9ebf6f\" (UID: \"f02602f2-1009-47b0-a05a-e69c2a9ebf6f\") "
Oct 30 22:47:28.289315 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:47:28.289291    4412 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f02602f2-1009-47b0-a05a-e69c2a9ebf6f-kserve-provision-location" pod "f02602f2-1009-47b0-a05a-e69c2a9ebf6f" (UID: "f02602f2-1009-47b0-a05a-e69c2a9ebf6f"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:47:28.289405 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:47:28.289295    4412 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f02602f2-1009-47b0-a05a-e69c2a9ebf6f-cabundle-cert" pod "f02602f2-1009-47b0-a05a-e69c2a9ebf6f" (UID: "f02602f2-1009-47b0-a05a-e69c2a9ebf6f"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:47:28.289497 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:47:28.289473    4412 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f02602f2-1009-47b0-a05a-e69c2a9ebf6f-kube-rbac-proxy-sar-config" pod "f02602f2-1009-47b0-a05a-e69c2a9ebf6f" (UID: "f02602f2-1009-47b0-a05a-e69c2a9ebf6f"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:47:28.291230 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:47:28.291208    4412 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f02602f2-1009-47b0-a05a-e69c2a9ebf6f-proxy-tls" pod "f02602f2-1009-47b0-a05a-e69c2a9ebf6f" (UID: "f02602f2-1009-47b0-a05a-e69c2a9ebf6f"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:47:28.291335 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:47:28.291294    4412 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f02602f2-1009-47b0-a05a-e69c2a9ebf6f-kube-api-access-xqnwd" pod "f02602f2-1009-47b0-a05a-e69c2a9ebf6f" (UID: "f02602f2-1009-47b0-a05a-e69c2a9ebf6f"). InnerVolumeSpecName "kube-api-access-xqnwd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:47:28.390061 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:47:28.390038    4412 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/f02602f2-1009-47b0-a05a-e69c2a9ebf6f-proxy-tls\") on node \"ip-10-0-131-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:47:28.390061 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:47:28.390058    4412 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xqnwd\" (UniqueName: \"kubernetes.io/projected/f02602f2-1009-47b0-a05a-e69c2a9ebf6f-kube-api-access-xqnwd\") on node \"ip-10-0-131-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:47:28.390241 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:47:28.390068    4412 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/f02602f2-1009-47b0-a05a-e69c2a9ebf6f-cabundle-cert\") on node \"ip-10-0-131-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:47:28.390241 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:47:28.390076    4412 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/f02602f2-1009-47b0-a05a-e69c2a9ebf6f-kserve-provision-location\") on node \"ip-10-0-131-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:47:28.390241 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:47:28.390085    4412 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/f02602f2-1009-47b0-a05a-e69c2a9ebf6f-kube-rbac-proxy-sar-config\") on node \"ip-10-0-131-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:47:28.686027 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:47:28.685948    4412 generic.go:356] "Generic (PLEG): container finished" podID="f02602f2-1009-47b0-a05a-e69c2a9ebf6f" containerID="24bb7c75fb96faebf5e9de8d9a5323607ae1ee20061ca7f96de19b3b3bd1c882" exitCode=0
Oct 30 22:47:28.686027 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:47:28.686008    4412 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-37d062d1dd/isvc-canary-stop-predictor-748fd79575-vr6cb"
Oct 30 22:47:28.686286 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:47:28.686031    4412 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-37d062d1dd/isvc-canary-stop-predictor-748fd79575-vr6cb" event={"ID":"f02602f2-1009-47b0-a05a-e69c2a9ebf6f","Type":"ContainerDied","Data":"24bb7c75fb96faebf5e9de8d9a5323607ae1ee20061ca7f96de19b3b3bd1c882"}
Oct 30 22:47:28.686286 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:47:28.686099    4412 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-37d062d1dd/isvc-canary-stop-predictor-748fd79575-vr6cb" event={"ID":"f02602f2-1009-47b0-a05a-e69c2a9ebf6f","Type":"ContainerDied","Data":"d8ecc1dbd12ddf37dc85edf3da898933ac4b19c2841ed65449134681651bca2d"}
Oct 30 22:47:28.686286 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:47:28.686153    4412 scope.go:122] "RemoveContainer" containerID="e267e8357ff576e471bc4b993510394dcc01aee681f812ef786a7768a8c9ff52"
Oct 30 22:47:28.694740 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:47:28.694723    4412 scope.go:122] "RemoveContainer" containerID="24bb7c75fb96faebf5e9de8d9a5323607ae1ee20061ca7f96de19b3b3bd1c882"
Oct 30 22:47:28.702050 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:47:28.702030    4412 scope.go:122] "RemoveContainer" containerID="48355f431b12a16b84f0a3aef909b2353c2e51b4f1e2a66310573aef2261372f"
Oct 30 22:47:28.708977 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:47:28.708960    4412 scope.go:122] "RemoveContainer" containerID="e267e8357ff576e471bc4b993510394dcc01aee681f812ef786a7768a8c9ff52"
Oct 30 22:47:28.709245 ip-10-0-131-202 kubenswrapper[4412]: E1030 22:47:28.709225    4412 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e267e8357ff576e471bc4b993510394dcc01aee681f812ef786a7768a8c9ff52\": container with ID starting with e267e8357ff576e471bc4b993510394dcc01aee681f812ef786a7768a8c9ff52 not found: ID does not exist" containerID="e267e8357ff576e471bc4b993510394dcc01aee681f812ef786a7768a8c9ff52"
Oct 30 22:47:28.709294 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:47:28.709255    4412 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e267e8357ff576e471bc4b993510394dcc01aee681f812ef786a7768a8c9ff52"} err="failed to get container status \"e267e8357ff576e471bc4b993510394dcc01aee681f812ef786a7768a8c9ff52\": rpc error: code = NotFound desc = could not find container \"e267e8357ff576e471bc4b993510394dcc01aee681f812ef786a7768a8c9ff52\": container with ID starting with e267e8357ff576e471bc4b993510394dcc01aee681f812ef786a7768a8c9ff52 not found: ID does not exist"
Oct 30 22:47:28.709294 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:47:28.709273    4412 scope.go:122] "RemoveContainer" containerID="24bb7c75fb96faebf5e9de8d9a5323607ae1ee20061ca7f96de19b3b3bd1c882"
Oct 30 22:47:28.709501 ip-10-0-131-202 kubenswrapper[4412]: E1030 22:47:28.709486    4412 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"24bb7c75fb96faebf5e9de8d9a5323607ae1ee20061ca7f96de19b3b3bd1c882\": container with ID starting with 24bb7c75fb96faebf5e9de8d9a5323607ae1ee20061ca7f96de19b3b3bd1c882 not found: ID does not exist" containerID="24bb7c75fb96faebf5e9de8d9a5323607ae1ee20061ca7f96de19b3b3bd1c882"
Oct 30 22:47:28.709541 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:47:28.709505    4412 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"24bb7c75fb96faebf5e9de8d9a5323607ae1ee20061ca7f96de19b3b3bd1c882"} err="failed to get container status \"24bb7c75fb96faebf5e9de8d9a5323607ae1ee20061ca7f96de19b3b3bd1c882\": rpc error: code = NotFound desc = could not find container \"24bb7c75fb96faebf5e9de8d9a5323607ae1ee20061ca7f96de19b3b3bd1c882\": container with ID starting with 24bb7c75fb96faebf5e9de8d9a5323607ae1ee20061ca7f96de19b3b3bd1c882 not found: ID does not exist"
Oct 30 22:47:28.709541 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:47:28.709516    4412 scope.go:122] "RemoveContainer" containerID="48355f431b12a16b84f0a3aef909b2353c2e51b4f1e2a66310573aef2261372f"
Oct 30 22:47:28.709730 ip-10-0-131-202 kubenswrapper[4412]: E1030 22:47:28.709716    4412 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"48355f431b12a16b84f0a3aef909b2353c2e51b4f1e2a66310573aef2261372f\": container with ID starting with 48355f431b12a16b84f0a3aef909b2353c2e51b4f1e2a66310573aef2261372f not found: ID does not exist" containerID="48355f431b12a16b84f0a3aef909b2353c2e51b4f1e2a66310573aef2261372f"
Oct 30 22:47:28.709781 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:47:28.709734    4412 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"48355f431b12a16b84f0a3aef909b2353c2e51b4f1e2a66310573aef2261372f"} err="failed to get container status \"48355f431b12a16b84f0a3aef909b2353c2e51b4f1e2a66310573aef2261372f\": rpc error: code = NotFound desc = could not find container \"48355f431b12a16b84f0a3aef909b2353c2e51b4f1e2a66310573aef2261372f\": container with ID starting with 48355f431b12a16b84f0a3aef909b2353c2e51b4f1e2a66310573aef2261372f not found: ID does not exist"
Oct 30 22:47:28.853591 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:47:28.853557    4412 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw0-37d062d1dd/isvc-canary-stop-predictor-748fd79575-vr6cb"]
Oct 30 22:47:28.903928 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:47:28.903889    4412 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-gw0-37d062d1dd/isvc-canary-stop-predictor-748fd79575-vr6cb"]
Oct 30 22:47:29.627172 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:47:29.627138    4412 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f02602f2-1009-47b0-a05a-e69c2a9ebf6f" path="/var/lib/kubelet/pods/f02602f2-1009-47b0-a05a-e69c2a9ebf6f/volumes"
Oct 30 22:47:34.448790 ip-10-0-131-202 kubenswrapper[4412]: E1030 22:47:34.448754    4412 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bkhmz" containerName="extractor"
Oct 30 22:48:04.447396 ip-10-0-131-202 kubenswrapper[4412]: E1030 22:48:04.447361    4412 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bkhmz" containerName="extractor"
Oct 30 22:48:34.449082 ip-10-0-131-202 kubenswrapper[4412]: E1030 22:48:34.449045    4412 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bkhmz" containerName="extractor"
Oct 30 22:49:04.448646 ip-10-0-131-202 kubenswrapper[4412]: E1030 22:49:04.448600    4412 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bkhmz" containerName="extractor"
Oct 30 22:49:30.715393 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:49:30.715357    4412 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-gw1-7476d02544/isvc-sklearn-s3-tls-serving-fail-predictor-5f56cf5887-429xg"]
Oct 30 22:49:30.715875 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:49:30.715655    4412 cpu_manager.go:414] "containerMap: removing container" podUID="f02602f2-1009-47b0-a05a-e69c2a9ebf6f" containerName="kube-rbac-proxy"
Oct 30 22:49:30.715875 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:49:30.715666    4412 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f02602f2-1009-47b0-a05a-e69c2a9ebf6f" containerName="kube-rbac-proxy"
Oct 30 22:49:30.715875 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:49:30.715694    4412 cpu_manager.go:414] "containerMap: removing container" podUID="f02602f2-1009-47b0-a05a-e69c2a9ebf6f" containerName="storage-initializer"
Oct 30 22:49:30.715875 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:49:30.715699    4412 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f02602f2-1009-47b0-a05a-e69c2a9ebf6f" containerName="storage-initializer"
Oct 30 22:49:30.715875 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:49:30.715706    4412 cpu_manager.go:414] "containerMap: removing container" podUID="f02602f2-1009-47b0-a05a-e69c2a9ebf6f" containerName="kserve-container"
Oct 30 22:49:30.715875 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:49:30.715711    4412 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f02602f2-1009-47b0-a05a-e69c2a9ebf6f" containerName="kserve-container"
Oct 30 22:49:30.715875 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:49:30.715753    4412 memory_manager.go:365] "RemoveStaleState removing state" podUID="f02602f2-1009-47b0-a05a-e69c2a9ebf6f" containerName="kserve-container"
Oct 30 22:49:30.715875 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:49:30.715762    4412 memory_manager.go:365] "RemoveStaleState removing state" podUID="f02602f2-1009-47b0-a05a-e69c2a9ebf6f" containerName="kube-rbac-proxy"
Oct 30 22:49:30.718697 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:49:30.718681    4412 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-7476d02544/isvc-sklearn-s3-tls-serving-fail-predictor-5f56cf5887-429xg"
Oct 30 22:49:30.814572 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:49:30.814511    4412 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/c82cc398-d370-45cf-a215-3dd620cea6b1-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-s3-tls-serving-fail-predictor-5f56cf5887-429xg\" (UID: \"c82cc398-d370-45cf-a215-3dd620cea6b1\") " pod="e2e-gw1-7476d02544/isvc-sklearn-s3-tls-serving-fail-predictor-5f56cf5887-429xg"
Oct 30 22:49:30.814572 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:49:30.814577    4412 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/c82cc398-d370-45cf-a215-3dd620cea6b1-proxy-tls\") pod \"isvc-sklearn-s3-tls-serving-fail-predictor-5f56cf5887-429xg\" (UID: \"c82cc398-d370-45cf-a215-3dd620cea6b1\") " pod="e2e-gw1-7476d02544/isvc-sklearn-s3-tls-serving-fail-predictor-5f56cf5887-429xg"
Oct 30 22:49:30.814853 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:49:30.814666    4412 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/c82cc398-d370-45cf-a215-3dd620cea6b1-kserve-provision-location\") pod \"isvc-sklearn-s3-tls-serving-fail-predictor-5f56cf5887-429xg\" (UID: \"c82cc398-d370-45cf-a215-3dd620cea6b1\") " pod="e2e-gw1-7476d02544/isvc-sklearn-s3-tls-serving-fail-predictor-5f56cf5887-429xg"
Oct 30 22:49:30.814853 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:49:30.814701    4412 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qdrbq\" (UniqueName: \"kubernetes.io/projected/c82cc398-d370-45cf-a215-3dd620cea6b1-kube-api-access-qdrbq\") pod \"isvc-sklearn-s3-tls-serving-fail-predictor-5f56cf5887-429xg\" (UID: \"c82cc398-d370-45cf-a215-3dd620cea6b1\") " pod="e2e-gw1-7476d02544/isvc-sklearn-s3-tls-serving-fail-predictor-5f56cf5887-429xg"
Oct 30 22:49:30.823359 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:49:30.823332    4412 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw1-7476d02544\"/\"isvc-sklearn-s3-tls-serving-fail-predictor-serving-cert\""
Oct 30 22:49:30.823573 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:49:30.823382    4412 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw1-7476d02544\"/\"kube-root-ca.crt\""
Oct 30 22:49:30.823573 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:49:30.823559    4412 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw1-7476d02544\"/\"openshift-service-ca.crt\""
Oct 30 22:49:30.823731 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:49:30.823445    4412 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw1-7476d02544\"/\"isvc-sklearn-s3-tls-serving-fail-kube-rbac-proxy-sar-config\""
Oct 30 22:49:30.823731 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:49:30.823480    4412 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw1-7476d02544\"/\"storage-config\""
Oct 30 22:49:30.823731 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:49:30.823422    4412 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw1-7476d02544\"/\"default-dockercfg-cxfkn\""
Oct 30 22:49:30.825736 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:49:30.825716    4412 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw1-7476d02544/isvc-sklearn-s3-tls-serving-fail-predictor-5f56cf5887-429xg"]
Oct 30 22:49:30.915237 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:49:30.915204    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/c82cc398-d370-45cf-a215-3dd620cea6b1-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-s3-tls-serving-fail-predictor-5f56cf5887-429xg\" (UID: \"c82cc398-d370-45cf-a215-3dd620cea6b1\") " pod="e2e-gw1-7476d02544/isvc-sklearn-s3-tls-serving-fail-predictor-5f56cf5887-429xg"
Oct 30 22:49:30.915465 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:49:30.915279    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/c82cc398-d370-45cf-a215-3dd620cea6b1-proxy-tls\") pod \"isvc-sklearn-s3-tls-serving-fail-predictor-5f56cf5887-429xg\" (UID: \"c82cc398-d370-45cf-a215-3dd620cea6b1\") " pod="e2e-gw1-7476d02544/isvc-sklearn-s3-tls-serving-fail-predictor-5f56cf5887-429xg"
Oct 30 22:49:30.915465 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:49:30.915304    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/c82cc398-d370-45cf-a215-3dd620cea6b1-kserve-provision-location\") pod \"isvc-sklearn-s3-tls-serving-fail-predictor-5f56cf5887-429xg\" (UID: \"c82cc398-d370-45cf-a215-3dd620cea6b1\") " pod="e2e-gw1-7476d02544/isvc-sklearn-s3-tls-serving-fail-predictor-5f56cf5887-429xg"
Oct 30 22:49:30.915465 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:49:30.915332    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qdrbq\" (UniqueName: \"kubernetes.io/projected/c82cc398-d370-45cf-a215-3dd620cea6b1-kube-api-access-qdrbq\") pod \"isvc-sklearn-s3-tls-serving-fail-predictor-5f56cf5887-429xg\" (UID: \"c82cc398-d370-45cf-a215-3dd620cea6b1\") " pod="e2e-gw1-7476d02544/isvc-sklearn-s3-tls-serving-fail-predictor-5f56cf5887-429xg"
Oct 30 22:49:30.915760 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:49:30.915732    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/c82cc398-d370-45cf-a215-3dd620cea6b1-kserve-provision-location\") pod \"isvc-sklearn-s3-tls-serving-fail-predictor-5f56cf5887-429xg\" (UID: \"c82cc398-d370-45cf-a215-3dd620cea6b1\") " pod="e2e-gw1-7476d02544/isvc-sklearn-s3-tls-serving-fail-predictor-5f56cf5887-429xg"
Oct 30 22:49:30.915918 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:49:30.915896    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/c82cc398-d370-45cf-a215-3dd620cea6b1-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-s3-tls-serving-fail-predictor-5f56cf5887-429xg\" (UID: \"c82cc398-d370-45cf-a215-3dd620cea6b1\") " pod="e2e-gw1-7476d02544/isvc-sklearn-s3-tls-serving-fail-predictor-5f56cf5887-429xg"
Oct 30 22:49:30.918078 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:49:30.918056    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/c82cc398-d370-45cf-a215-3dd620cea6b1-proxy-tls\") pod \"isvc-sklearn-s3-tls-serving-fail-predictor-5f56cf5887-429xg\" (UID: \"c82cc398-d370-45cf-a215-3dd620cea6b1\") " pod="e2e-gw1-7476d02544/isvc-sklearn-s3-tls-serving-fail-predictor-5f56cf5887-429xg"
Oct 30 22:49:30.973786 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:49:30.973714    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qdrbq\" (UniqueName: \"kubernetes.io/projected/c82cc398-d370-45cf-a215-3dd620cea6b1-kube-api-access-qdrbq\") pod \"isvc-sklearn-s3-tls-serving-fail-predictor-5f56cf5887-429xg\" (UID: \"c82cc398-d370-45cf-a215-3dd620cea6b1\") " pod="e2e-gw1-7476d02544/isvc-sklearn-s3-tls-serving-fail-predictor-5f56cf5887-429xg"
Oct 30 22:49:31.034732 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:49:31.034694    4412 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-7476d02544/isvc-sklearn-s3-tls-serving-fail-predictor-5f56cf5887-429xg"
Oct 30 22:49:31.455611 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:49:31.455588    4412 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw1-7476d02544/isvc-sklearn-s3-tls-serving-fail-predictor-5f56cf5887-429xg"]
Oct 30 22:49:31.458195 ip-10-0-131-202 kubenswrapper[4412]: W1030 22:49:31.458165    4412 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc82cc398_d370_45cf_a215_3dd620cea6b1.slice/crio-b4932745c18c93dd2c8e0737dfdd922e47892ce8e4db6af48278a2ddc932ea19 WatchSource:0}: Error finding container b4932745c18c93dd2c8e0737dfdd922e47892ce8e4db6af48278a2ddc932ea19: Status 404 returned error can't find the container with id b4932745c18c93dd2c8e0737dfdd922e47892ce8e4db6af48278a2ddc932ea19
Oct 30 22:49:32.119442 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:49:32.119408    4412 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-7476d02544/isvc-sklearn-s3-tls-serving-fail-predictor-5f56cf5887-429xg" event={"ID":"c82cc398-d370-45cf-a215-3dd620cea6b1","Type":"ContainerStarted","Data":"a6eb52ab1aae5148df63a9029f3828899170f23327e282f7c4a19adbe4ef5fb4"}
Oct 30 22:49:32.119943 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:49:32.119475    4412 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-7476d02544/isvc-sklearn-s3-tls-serving-fail-predictor-5f56cf5887-429xg" event={"ID":"c82cc398-d370-45cf-a215-3dd620cea6b1","Type":"ContainerStarted","Data":"b4932745c18c93dd2c8e0737dfdd922e47892ce8e4db6af48278a2ddc932ea19"}
Oct 30 22:49:34.448608 ip-10-0-131-202 kubenswrapper[4412]: E1030 22:49:34.448572    4412 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bkhmz" containerName="extractor"
Oct 30 22:49:36.136939 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:49:36.136908    4412 log.go:25] "Finished parsing log file" path="/var/log/pods/e2e-gw1-7476d02544_isvc-sklearn-s3-tls-serving-fail-predictor-5f56cf5887-429xg_c82cc398-d370-45cf-a215-3dd620cea6b1/storage-initializer/0.log"
Oct 30 22:49:36.137380 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:49:36.136950    4412 generic.go:356] "Generic (PLEG): container finished" podID="c82cc398-d370-45cf-a215-3dd620cea6b1" containerID="a6eb52ab1aae5148df63a9029f3828899170f23327e282f7c4a19adbe4ef5fb4" exitCode=1
Oct 30 22:49:36.137380 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:49:36.137024    4412 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-7476d02544/isvc-sklearn-s3-tls-serving-fail-predictor-5f56cf5887-429xg" event={"ID":"c82cc398-d370-45cf-a215-3dd620cea6b1","Type":"ContainerDied","Data":"a6eb52ab1aae5148df63a9029f3828899170f23327e282f7c4a19adbe4ef5fb4"}
Oct 30 22:49:37.142143 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:49:37.142094    4412 log.go:25] "Finished parsing log file" path="/var/log/pods/e2e-gw1-7476d02544_isvc-sklearn-s3-tls-serving-fail-predictor-5f56cf5887-429xg_c82cc398-d370-45cf-a215-3dd620cea6b1/storage-initializer/0.log"
Oct 30 22:49:37.142535 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:49:37.142165    4412 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-7476d02544/isvc-sklearn-s3-tls-serving-fail-predictor-5f56cf5887-429xg" event={"ID":"c82cc398-d370-45cf-a215-3dd620cea6b1","Type":"ContainerStarted","Data":"91814fc7747c653c94b41ec4222951a0e31709151834b35ee56d989231e180d2"}
Oct 30 22:49:40.248154 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:49:40.248104    4412 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw1-7476d02544/isvc-sklearn-s3-tls-serving-fail-predictor-5f56cf5887-429xg"]
Oct 30 22:49:40.248548 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:49:40.248362    4412 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw1-7476d02544/isvc-sklearn-s3-tls-serving-fail-predictor-5f56cf5887-429xg" podUID="c82cc398-d370-45cf-a215-3dd620cea6b1" containerName="storage-initializer" containerID="cri-o://91814fc7747c653c94b41ec4222951a0e31709151834b35ee56d989231e180d2" gracePeriod=30
Oct 30 22:49:41.582271 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:49:41.582247    4412 log.go:25] "Finished parsing log file" path="/var/log/pods/e2e-gw1-7476d02544_isvc-sklearn-s3-tls-serving-fail-predictor-5f56cf5887-429xg_c82cc398-d370-45cf-a215-3dd620cea6b1/storage-initializer/1.log"
Oct 30 22:49:41.582669 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:49:41.582655    4412 log.go:25] "Finished parsing log file" path="/var/log/pods/e2e-gw1-7476d02544_isvc-sklearn-s3-tls-serving-fail-predictor-5f56cf5887-429xg_c82cc398-d370-45cf-a215-3dd620cea6b1/storage-initializer/0.log"
Oct 30 22:49:41.584902 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:49:41.584888    4412 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-7476d02544/isvc-sklearn-s3-tls-serving-fail-predictor-5f56cf5887-429xg"
Oct 30 22:49:41.591939 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:49:41.591921    4412 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/c82cc398-d370-45cf-a215-3dd620cea6b1-proxy-tls\" (UniqueName: \"kubernetes.io/secret/c82cc398-d370-45cf-a215-3dd620cea6b1-proxy-tls\") pod \"c82cc398-d370-45cf-a215-3dd620cea6b1\" (UID: \"c82cc398-d370-45cf-a215-3dd620cea6b1\") "
Oct 30 22:49:41.592029 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:49:41.591975    4412 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c82cc398-d370-45cf-a215-3dd620cea6b1-kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/c82cc398-d370-45cf-a215-3dd620cea6b1-kube-rbac-proxy-sar-config\") pod \"c82cc398-d370-45cf-a215-3dd620cea6b1\" (UID: \"c82cc398-d370-45cf-a215-3dd620cea6b1\") "
Oct 30 22:49:41.592029 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:49:41.592020    4412 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c82cc398-d370-45cf-a215-3dd620cea6b1-kube-api-access-qdrbq\" (UniqueName: \"kubernetes.io/projected/c82cc398-d370-45cf-a215-3dd620cea6b1-kube-api-access-qdrbq\") pod \"c82cc398-d370-45cf-a215-3dd620cea6b1\" (UID: \"c82cc398-d370-45cf-a215-3dd620cea6b1\") "
Oct 30 22:49:41.592164 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:49:41.592068    4412 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c82cc398-d370-45cf-a215-3dd620cea6b1-kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/c82cc398-d370-45cf-a215-3dd620cea6b1-kserve-provision-location\") pod \"c82cc398-d370-45cf-a215-3dd620cea6b1\" (UID: \"c82cc398-d370-45cf-a215-3dd620cea6b1\") "
Oct 30 22:49:41.592351 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:49:41.592325    4412 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c82cc398-d370-45cf-a215-3dd620cea6b1-kserve-provision-location" pod "c82cc398-d370-45cf-a215-3dd620cea6b1" (UID: "c82cc398-d370-45cf-a215-3dd620cea6b1"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:49:41.592351 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:49:41.592343    4412 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c82cc398-d370-45cf-a215-3dd620cea6b1-kube-rbac-proxy-sar-config" pod "c82cc398-d370-45cf-a215-3dd620cea6b1" (UID: "c82cc398-d370-45cf-a215-3dd620cea6b1"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:49:41.594261 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:49:41.594240    4412 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c82cc398-d370-45cf-a215-3dd620cea6b1-proxy-tls" pod "c82cc398-d370-45cf-a215-3dd620cea6b1" (UID: "c82cc398-d370-45cf-a215-3dd620cea6b1"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:49:41.594347 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:49:41.594286    4412 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c82cc398-d370-45cf-a215-3dd620cea6b1-kube-api-access-qdrbq" pod "c82cc398-d370-45cf-a215-3dd620cea6b1" (UID: "c82cc398-d370-45cf-a215-3dd620cea6b1"). InnerVolumeSpecName "kube-api-access-qdrbq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:49:41.693393 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:49:41.693313    4412 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/c82cc398-d370-45cf-a215-3dd620cea6b1-kube-rbac-proxy-sar-config\") on node \"ip-10-0-131-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:49:41.693393 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:49:41.693339    4412 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qdrbq\" (UniqueName: \"kubernetes.io/projected/c82cc398-d370-45cf-a215-3dd620cea6b1-kube-api-access-qdrbq\") on node \"ip-10-0-131-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:49:41.693393 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:49:41.693349    4412 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/c82cc398-d370-45cf-a215-3dd620cea6b1-kserve-provision-location\") on node \"ip-10-0-131-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:49:41.693393 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:49:41.693357    4412 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/c82cc398-d370-45cf-a215-3dd620cea6b1-proxy-tls\") on node \"ip-10-0-131-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:49:42.161612 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:49:42.161587    4412 log.go:25] "Finished parsing log file" path="/var/log/pods/e2e-gw1-7476d02544_isvc-sklearn-s3-tls-serving-fail-predictor-5f56cf5887-429xg_c82cc398-d370-45cf-a215-3dd620cea6b1/storage-initializer/1.log"
Oct 30 22:49:42.161991 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:49:42.161973    4412 log.go:25] "Finished parsing log file" path="/var/log/pods/e2e-gw1-7476d02544_isvc-sklearn-s3-tls-serving-fail-predictor-5f56cf5887-429xg_c82cc398-d370-45cf-a215-3dd620cea6b1/storage-initializer/0.log"
Oct 30 22:49:42.162114 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:49:42.162014    4412 generic.go:356] "Generic (PLEG): container finished" podID="c82cc398-d370-45cf-a215-3dd620cea6b1" containerID="91814fc7747c653c94b41ec4222951a0e31709151834b35ee56d989231e180d2" exitCode=1
Oct 30 22:49:42.162114 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:49:42.162072    4412 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-7476d02544/isvc-sklearn-s3-tls-serving-fail-predictor-5f56cf5887-429xg"
Oct 30 22:49:42.162114 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:49:42.162079    4412 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-7476d02544/isvc-sklearn-s3-tls-serving-fail-predictor-5f56cf5887-429xg" event={"ID":"c82cc398-d370-45cf-a215-3dd620cea6b1","Type":"ContainerDied","Data":"91814fc7747c653c94b41ec4222951a0e31709151834b35ee56d989231e180d2"}
Oct 30 22:49:42.162315 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:49:42.162162    4412 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-7476d02544/isvc-sklearn-s3-tls-serving-fail-predictor-5f56cf5887-429xg" event={"ID":"c82cc398-d370-45cf-a215-3dd620cea6b1","Type":"ContainerDied","Data":"b4932745c18c93dd2c8e0737dfdd922e47892ce8e4db6af48278a2ddc932ea19"}
Oct 30 22:49:42.162315 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:49:42.162207    4412 scope.go:122] "RemoveContainer" containerID="91814fc7747c653c94b41ec4222951a0e31709151834b35ee56d989231e180d2"
Oct 30 22:49:42.170283 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:49:42.170252    4412 scope.go:122] "RemoveContainer" containerID="a6eb52ab1aae5148df63a9029f3828899170f23327e282f7c4a19adbe4ef5fb4"
Oct 30 22:49:42.177483 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:49:42.177460    4412 scope.go:122] "RemoveContainer" containerID="91814fc7747c653c94b41ec4222951a0e31709151834b35ee56d989231e180d2"
Oct 30 22:49:42.177777 ip-10-0-131-202 kubenswrapper[4412]: E1030 22:49:42.177751    4412 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"91814fc7747c653c94b41ec4222951a0e31709151834b35ee56d989231e180d2\": container with ID starting with 91814fc7747c653c94b41ec4222951a0e31709151834b35ee56d989231e180d2 not found: ID does not exist" containerID="91814fc7747c653c94b41ec4222951a0e31709151834b35ee56d989231e180d2"
Oct 30 22:49:42.177837 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:49:42.177786    4412 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"91814fc7747c653c94b41ec4222951a0e31709151834b35ee56d989231e180d2"} err="failed to get container status \"91814fc7747c653c94b41ec4222951a0e31709151834b35ee56d989231e180d2\": rpc error: code = NotFound desc = could not find container \"91814fc7747c653c94b41ec4222951a0e31709151834b35ee56d989231e180d2\": container with ID starting with 91814fc7747c653c94b41ec4222951a0e31709151834b35ee56d989231e180d2 not found: ID does not exist"
Oct 30 22:49:42.177837 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:49:42.177805    4412 scope.go:122] "RemoveContainer" containerID="a6eb52ab1aae5148df63a9029f3828899170f23327e282f7c4a19adbe4ef5fb4"
Oct 30 22:49:42.178086 ip-10-0-131-202 kubenswrapper[4412]: E1030 22:49:42.178068    4412 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a6eb52ab1aae5148df63a9029f3828899170f23327e282f7c4a19adbe4ef5fb4\": container with ID starting with a6eb52ab1aae5148df63a9029f3828899170f23327e282f7c4a19adbe4ef5fb4 not found: ID does not exist" containerID="a6eb52ab1aae5148df63a9029f3828899170f23327e282f7c4a19adbe4ef5fb4"
Oct 30 22:49:42.178163 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:49:42.178092    4412 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a6eb52ab1aae5148df63a9029f3828899170f23327e282f7c4a19adbe4ef5fb4"} err="failed to get container status \"a6eb52ab1aae5148df63a9029f3828899170f23327e282f7c4a19adbe4ef5fb4\": rpc error: code = NotFound desc = could not find container \"a6eb52ab1aae5148df63a9029f3828899170f23327e282f7c4a19adbe4ef5fb4\": container with ID starting with a6eb52ab1aae5148df63a9029f3828899170f23327e282f7c4a19adbe4ef5fb4 not found: ID does not exist"
Oct 30 22:49:42.394186 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:49:42.394148    4412 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw1-7476d02544/isvc-sklearn-s3-tls-serving-fail-predictor-5f56cf5887-429xg"]
Oct 30 22:49:42.444371 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:49:42.444291    4412 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-gw1-7476d02544/isvc-sklearn-s3-tls-serving-fail-predictor-5f56cf5887-429xg"]
Oct 30 22:49:43.626468 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:49:43.626433    4412 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c82cc398-d370-45cf-a215-3dd620cea6b1" path="/var/lib/kubelet/pods/c82cc398-d370-45cf-a215-3dd620cea6b1/volumes"
Oct 30 22:49:45.271178 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:49:45.271144    4412 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-gw1-7476d02544/isvc-paddle-predictor-74cbfb4bcb-d9zph"]
Oct 30 22:49:45.271582 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:49:45.271414    4412 cpu_manager.go:414] "containerMap: removing container" podUID="c82cc398-d370-45cf-a215-3dd620cea6b1" containerName="storage-initializer"
Oct 30 22:49:45.271582 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:49:45.271425    4412 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c82cc398-d370-45cf-a215-3dd620cea6b1" containerName="storage-initializer"
Oct 30 22:49:45.271582 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:49:45.271446    4412 cpu_manager.go:414] "containerMap: removing container" podUID="c82cc398-d370-45cf-a215-3dd620cea6b1" containerName="storage-initializer"
Oct 30 22:49:45.271582 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:49:45.271452    4412 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c82cc398-d370-45cf-a215-3dd620cea6b1" containerName="storage-initializer"
Oct 30 22:49:45.271582 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:49:45.271504    4412 memory_manager.go:365] "RemoveStaleState removing state" podUID="c82cc398-d370-45cf-a215-3dd620cea6b1" containerName="storage-initializer"
Oct 30 22:49:45.271731 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:49:45.271594    4412 memory_manager.go:365] "RemoveStaleState removing state" podUID="c82cc398-d370-45cf-a215-3dd620cea6b1" containerName="storage-initializer"
Oct 30 22:49:45.275691 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:49:45.275675    4412 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-7476d02544/isvc-paddle-predictor-74cbfb4bcb-d9zph"
Oct 30 22:49:45.320709 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:49:45.320667    4412 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/0be79600-f0e1-4f26-bf22-dec5d2079875-cabundle-cert\") pod \"isvc-paddle-predictor-74cbfb4bcb-d9zph\" (UID: \"0be79600-f0e1-4f26-bf22-dec5d2079875\") " pod="e2e-gw1-7476d02544/isvc-paddle-predictor-74cbfb4bcb-d9zph"
Oct 30 22:49:45.320877 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:49:45.320721    4412 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8dhn4\" (UniqueName: \"kubernetes.io/projected/0be79600-f0e1-4f26-bf22-dec5d2079875-kube-api-access-8dhn4\") pod \"isvc-paddle-predictor-74cbfb4bcb-d9zph\" (UID: \"0be79600-f0e1-4f26-bf22-dec5d2079875\") " pod="e2e-gw1-7476d02544/isvc-paddle-predictor-74cbfb4bcb-d9zph"
Oct 30 22:49:45.320877 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:49:45.320847    4412 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/0be79600-f0e1-4f26-bf22-dec5d2079875-proxy-tls\") pod \"isvc-paddle-predictor-74cbfb4bcb-d9zph\" (UID: \"0be79600-f0e1-4f26-bf22-dec5d2079875\") " pod="e2e-gw1-7476d02544/isvc-paddle-predictor-74cbfb4bcb-d9zph"
Oct 30 22:49:45.320966 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:49:45.320881    4412 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/0be79600-f0e1-4f26-bf22-dec5d2079875-kube-rbac-proxy-sar-config\") pod \"isvc-paddle-predictor-74cbfb4bcb-d9zph\" (UID: \"0be79600-f0e1-4f26-bf22-dec5d2079875\") " pod="e2e-gw1-7476d02544/isvc-paddle-predictor-74cbfb4bcb-d9zph"
Oct 30 22:49:45.321007 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:49:45.320958    4412 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/0be79600-f0e1-4f26-bf22-dec5d2079875-kserve-provision-location\") pod \"isvc-paddle-predictor-74cbfb4bcb-d9zph\" (UID: \"0be79600-f0e1-4f26-bf22-dec5d2079875\") " pod="e2e-gw1-7476d02544/isvc-paddle-predictor-74cbfb4bcb-d9zph"
Oct 30 22:49:45.329664 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:49:45.329633    4412 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw1-7476d02544\"/\"kube-root-ca.crt\""
Oct 30 22:49:45.329664 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:49:45.329640    4412 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw1-7476d02544\"/\"seaweedfs-s3-creds\""
Oct 30 22:49:45.329664 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:49:45.329661    4412 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw1-7476d02544\"/\"default-dockercfg-cxfkn\""
Oct 30 22:49:45.380340 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:49:45.380311    4412 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw1-7476d02544\"/\"openshift-service-ca.crt\""
Oct 30 22:49:45.380539 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:49:45.380311    4412 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw1-7476d02544\"/\"odh-kserve-custom-ca-bundle\""
Oct 30 22:49:45.380539 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:49:45.380367    4412 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw1-7476d02544\"/\"isvc-paddle-kube-rbac-proxy-sar-config\""
Oct 30 22:49:45.380539 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:49:45.380373    4412 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw1-7476d02544\"/\"isvc-paddle-predictor-serving-cert\""
Oct 30 22:49:45.382303 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:49:45.382283    4412 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw1-7476d02544/isvc-paddle-predictor-74cbfb4bcb-d9zph"]
Oct 30 22:49:45.421598 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:49:45.421573    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/0be79600-f0e1-4f26-bf22-dec5d2079875-cabundle-cert\") pod \"isvc-paddle-predictor-74cbfb4bcb-d9zph\" (UID: \"0be79600-f0e1-4f26-bf22-dec5d2079875\") " pod="e2e-gw1-7476d02544/isvc-paddle-predictor-74cbfb4bcb-d9zph"
Oct 30 22:49:45.421751 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:49:45.421610    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8dhn4\" (UniqueName: \"kubernetes.io/projected/0be79600-f0e1-4f26-bf22-dec5d2079875-kube-api-access-8dhn4\") pod \"isvc-paddle-predictor-74cbfb4bcb-d9zph\" (UID: \"0be79600-f0e1-4f26-bf22-dec5d2079875\") " pod="e2e-gw1-7476d02544/isvc-paddle-predictor-74cbfb4bcb-d9zph"
Oct 30 22:49:45.421751 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:49:45.421641    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/0be79600-f0e1-4f26-bf22-dec5d2079875-proxy-tls\") pod \"isvc-paddle-predictor-74cbfb4bcb-d9zph\" (UID: \"0be79600-f0e1-4f26-bf22-dec5d2079875\") " pod="e2e-gw1-7476d02544/isvc-paddle-predictor-74cbfb4bcb-d9zph"
Oct 30 22:49:45.421751 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:49:45.421667    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/0be79600-f0e1-4f26-bf22-dec5d2079875-kube-rbac-proxy-sar-config\") pod \"isvc-paddle-predictor-74cbfb4bcb-d9zph\" (UID: \"0be79600-f0e1-4f26-bf22-dec5d2079875\") " pod="e2e-gw1-7476d02544/isvc-paddle-predictor-74cbfb4bcb-d9zph"
Oct 30 22:49:45.421751 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:49:45.421700    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/0be79600-f0e1-4f26-bf22-dec5d2079875-kserve-provision-location\") pod \"isvc-paddle-predictor-74cbfb4bcb-d9zph\" (UID: \"0be79600-f0e1-4f26-bf22-dec5d2079875\") " pod="e2e-gw1-7476d02544/isvc-paddle-predictor-74cbfb4bcb-d9zph"
Oct 30 22:49:45.422158 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:49:45.422115    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/0be79600-f0e1-4f26-bf22-dec5d2079875-kserve-provision-location\") pod \"isvc-paddle-predictor-74cbfb4bcb-d9zph\" (UID: \"0be79600-f0e1-4f26-bf22-dec5d2079875\") " pod="e2e-gw1-7476d02544/isvc-paddle-predictor-74cbfb4bcb-d9zph"
Oct 30 22:49:45.422371 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:49:45.422355    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/0be79600-f0e1-4f26-bf22-dec5d2079875-cabundle-cert\") pod \"isvc-paddle-predictor-74cbfb4bcb-d9zph\" (UID: \"0be79600-f0e1-4f26-bf22-dec5d2079875\") " pod="e2e-gw1-7476d02544/isvc-paddle-predictor-74cbfb4bcb-d9zph"
Oct 30 22:49:45.422415 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:49:45.422352    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/0be79600-f0e1-4f26-bf22-dec5d2079875-kube-rbac-proxy-sar-config\") pod \"isvc-paddle-predictor-74cbfb4bcb-d9zph\" (UID: \"0be79600-f0e1-4f26-bf22-dec5d2079875\") " pod="e2e-gw1-7476d02544/isvc-paddle-predictor-74cbfb4bcb-d9zph"
Oct 30 22:49:45.424364 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:49:45.424348    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/0be79600-f0e1-4f26-bf22-dec5d2079875-proxy-tls\") pod \"isvc-paddle-predictor-74cbfb4bcb-d9zph\" (UID: \"0be79600-f0e1-4f26-bf22-dec5d2079875\") " pod="e2e-gw1-7476d02544/isvc-paddle-predictor-74cbfb4bcb-d9zph"
Oct 30 22:49:45.479088 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:49:45.479049    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8dhn4\" (UniqueName: \"kubernetes.io/projected/0be79600-f0e1-4f26-bf22-dec5d2079875-kube-api-access-8dhn4\") pod \"isvc-paddle-predictor-74cbfb4bcb-d9zph\" (UID: \"0be79600-f0e1-4f26-bf22-dec5d2079875\") " pod="e2e-gw1-7476d02544/isvc-paddle-predictor-74cbfb4bcb-d9zph"
Oct 30 22:49:45.592401 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:49:45.592315    4412 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-7476d02544/isvc-paddle-predictor-74cbfb4bcb-d9zph"
Oct 30 22:49:46.013969 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:49:46.013943    4412 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw1-7476d02544/isvc-paddle-predictor-74cbfb4bcb-d9zph"]
Oct 30 22:49:46.016372 ip-10-0-131-202 kubenswrapper[4412]: W1030 22:49:46.016347    4412 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0be79600_f0e1_4f26_bf22_dec5d2079875.slice/crio-daa74435d2df5e27fb3550c70d7438d31514f7095bce289f7c89c1686abc9ede WatchSource:0}: Error finding container daa74435d2df5e27fb3550c70d7438d31514f7095bce289f7c89c1686abc9ede: Status 404 returned error can't find the container with id daa74435d2df5e27fb3550c70d7438d31514f7095bce289f7c89c1686abc9ede
Oct 30 22:49:46.186250 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:49:46.186217    4412 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-7476d02544/isvc-paddle-predictor-74cbfb4bcb-d9zph" event={"ID":"0be79600-f0e1-4f26-bf22-dec5d2079875","Type":"ContainerStarted","Data":"79d8e19e48556976e586e9cc4e5e6e47fdc99d3f284cf3a767c934cbe8fcc8d2"}
Oct 30 22:49:46.186551 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:49:46.186273    4412 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-7476d02544/isvc-paddle-predictor-74cbfb4bcb-d9zph" event={"ID":"0be79600-f0e1-4f26-bf22-dec5d2079875","Type":"ContainerStarted","Data":"daa74435d2df5e27fb3550c70d7438d31514f7095bce289f7c89c1686abc9ede"}
Oct 30 22:49:46.585615 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:49:46.585572    4412 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-gw0-37d062d1dd/isvc-lightgbm-runtime-predictor-5c98bb5cc5-m85nn"]
Oct 30 22:49:46.589710 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:49:46.589689    4412 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-37d062d1dd/isvc-lightgbm-runtime-predictor-5c98bb5cc5-m85nn"
Oct 30 22:49:46.630623 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:49:46.630597    4412 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/b5645218-816f-402f-92be-184fafc6e4de-kube-rbac-proxy-sar-config\") pod \"isvc-lightgbm-runtime-predictor-5c98bb5cc5-m85nn\" (UID: \"b5645218-816f-402f-92be-184fafc6e4de\") " pod="e2e-gw0-37d062d1dd/isvc-lightgbm-runtime-predictor-5c98bb5cc5-m85nn"
Oct 30 22:49:46.630779 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:49:46.630628    4412 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/b5645218-816f-402f-92be-184fafc6e4de-cabundle-cert\") pod \"isvc-lightgbm-runtime-predictor-5c98bb5cc5-m85nn\" (UID: \"b5645218-816f-402f-92be-184fafc6e4de\") " pod="e2e-gw0-37d062d1dd/isvc-lightgbm-runtime-predictor-5c98bb5cc5-m85nn"
Oct 30 22:49:46.630779 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:49:46.630653    4412 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sww6q\" (UniqueName: \"kubernetes.io/projected/b5645218-816f-402f-92be-184fafc6e4de-kube-api-access-sww6q\") pod \"isvc-lightgbm-runtime-predictor-5c98bb5cc5-m85nn\" (UID: \"b5645218-816f-402f-92be-184fafc6e4de\") " pod="e2e-gw0-37d062d1dd/isvc-lightgbm-runtime-predictor-5c98bb5cc5-m85nn"
Oct 30 22:49:46.630779 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:49:46.630714    4412 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/b5645218-816f-402f-92be-184fafc6e4de-kserve-provision-location\") pod \"isvc-lightgbm-runtime-predictor-5c98bb5cc5-m85nn\" (UID: \"b5645218-816f-402f-92be-184fafc6e4de\") " pod="e2e-gw0-37d062d1dd/isvc-lightgbm-runtime-predictor-5c98bb5cc5-m85nn"
Oct 30 22:49:46.630912 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:49:46.630883    4412 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/b5645218-816f-402f-92be-184fafc6e4de-proxy-tls\") pod \"isvc-lightgbm-runtime-predictor-5c98bb5cc5-m85nn\" (UID: \"b5645218-816f-402f-92be-184fafc6e4de\") " pod="e2e-gw0-37d062d1dd/isvc-lightgbm-runtime-predictor-5c98bb5cc5-m85nn"
Oct 30 22:49:46.643748 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:49:46.643726    4412 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw0-37d062d1dd\"/\"openshift-service-ca.crt\""
Oct 30 22:49:46.643748 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:49:46.643726    4412 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw0-37d062d1dd\"/\"seaweedfs-s3-creds\""
Oct 30 22:49:46.694136 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:49:46.694099    4412 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw0-37d062d1dd\"/\"odh-kserve-custom-ca-bundle\""
Oct 30 22:49:46.694367 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:49:46.694207    4412 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw0-37d062d1dd\"/\"isvc-lightgbm-runtime-predictor-serving-cert\""
Oct 30 22:49:46.694367 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:49:46.694254    4412 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw0-37d062d1dd\"/\"isvc-lightgbm-runtime-kube-rbac-proxy-sar-config\""
Oct 30 22:49:46.694367 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:49:46.694252    4412 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw0-37d062d1dd\"/\"kube-root-ca.crt\""
Oct 30 22:49:46.694367 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:49:46.694286    4412 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw0-37d062d1dd\"/\"default-dockercfg-4v4s8\""
Oct 30 22:49:46.695657 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:49:46.695636    4412 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw0-37d062d1dd/isvc-lightgbm-runtime-predictor-5c98bb5cc5-m85nn"]
Oct 30 22:49:46.731681 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:49:46.731650    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sww6q\" (UniqueName: \"kubernetes.io/projected/b5645218-816f-402f-92be-184fafc6e4de-kube-api-access-sww6q\") pod \"isvc-lightgbm-runtime-predictor-5c98bb5cc5-m85nn\" (UID: \"b5645218-816f-402f-92be-184fafc6e4de\") " pod="e2e-gw0-37d062d1dd/isvc-lightgbm-runtime-predictor-5c98bb5cc5-m85nn"
Oct 30 22:49:46.731809 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:49:46.731694    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/b5645218-816f-402f-92be-184fafc6e4de-kserve-provision-location\") pod \"isvc-lightgbm-runtime-predictor-5c98bb5cc5-m85nn\" (UID: \"b5645218-816f-402f-92be-184fafc6e4de\") " pod="e2e-gw0-37d062d1dd/isvc-lightgbm-runtime-predictor-5c98bb5cc5-m85nn"
Oct 30 22:49:46.731809 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:49:46.731750    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/b5645218-816f-402f-92be-184fafc6e4de-proxy-tls\") pod \"isvc-lightgbm-runtime-predictor-5c98bb5cc5-m85nn\" (UID: \"b5645218-816f-402f-92be-184fafc6e4de\") " pod="e2e-gw0-37d062d1dd/isvc-lightgbm-runtime-predictor-5c98bb5cc5-m85nn"
Oct 30 22:49:46.731809 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:49:46.731770    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/b5645218-816f-402f-92be-184fafc6e4de-kube-rbac-proxy-sar-config\") pod \"isvc-lightgbm-runtime-predictor-5c98bb5cc5-m85nn\" (UID: \"b5645218-816f-402f-92be-184fafc6e4de\") " pod="e2e-gw0-37d062d1dd/isvc-lightgbm-runtime-predictor-5c98bb5cc5-m85nn"
Oct 30 22:49:46.731809 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:49:46.731791    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/b5645218-816f-402f-92be-184fafc6e4de-cabundle-cert\") pod \"isvc-lightgbm-runtime-predictor-5c98bb5cc5-m85nn\" (UID: \"b5645218-816f-402f-92be-184fafc6e4de\") " pod="e2e-gw0-37d062d1dd/isvc-lightgbm-runtime-predictor-5c98bb5cc5-m85nn"
Oct 30 22:49:46.732222 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:49:46.732202    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/b5645218-816f-402f-92be-184fafc6e4de-kserve-provision-location\") pod \"isvc-lightgbm-runtime-predictor-5c98bb5cc5-m85nn\" (UID: \"b5645218-816f-402f-92be-184fafc6e4de\") " pod="e2e-gw0-37d062d1dd/isvc-lightgbm-runtime-predictor-5c98bb5cc5-m85nn"
Oct 30 22:49:46.732484 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:49:46.732463    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/b5645218-816f-402f-92be-184fafc6e4de-kube-rbac-proxy-sar-config\") pod \"isvc-lightgbm-runtime-predictor-5c98bb5cc5-m85nn\" (UID: \"b5645218-816f-402f-92be-184fafc6e4de\") " pod="e2e-gw0-37d062d1dd/isvc-lightgbm-runtime-predictor-5c98bb5cc5-m85nn"
Oct 30 22:49:46.732555 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:49:46.732520    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/b5645218-816f-402f-92be-184fafc6e4de-cabundle-cert\") pod \"isvc-lightgbm-runtime-predictor-5c98bb5cc5-m85nn\" (UID: \"b5645218-816f-402f-92be-184fafc6e4de\") " pod="e2e-gw0-37d062d1dd/isvc-lightgbm-runtime-predictor-5c98bb5cc5-m85nn"
Oct 30 22:49:46.734370 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:49:46.734353    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/b5645218-816f-402f-92be-184fafc6e4de-proxy-tls\") pod \"isvc-lightgbm-runtime-predictor-5c98bb5cc5-m85nn\" (UID: \"b5645218-816f-402f-92be-184fafc6e4de\") " pod="e2e-gw0-37d062d1dd/isvc-lightgbm-runtime-predictor-5c98bb5cc5-m85nn"
Oct 30 22:49:46.790330 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:49:46.790295    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sww6q\" (UniqueName: \"kubernetes.io/projected/b5645218-816f-402f-92be-184fafc6e4de-kube-api-access-sww6q\") pod \"isvc-lightgbm-runtime-predictor-5c98bb5cc5-m85nn\" (UID: \"b5645218-816f-402f-92be-184fafc6e4de\") " pod="e2e-gw0-37d062d1dd/isvc-lightgbm-runtime-predictor-5c98bb5cc5-m85nn"
Oct 30 22:49:46.906986 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:49:46.906894    4412 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-37d062d1dd/isvc-lightgbm-runtime-predictor-5c98bb5cc5-m85nn"
Oct 30 22:49:47.330606 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:49:47.330584    4412 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw0-37d062d1dd/isvc-lightgbm-runtime-predictor-5c98bb5cc5-m85nn"]
Oct 30 22:49:47.332358 ip-10-0-131-202 kubenswrapper[4412]: W1030 22:49:47.332330    4412 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb5645218_816f_402f_92be_184fafc6e4de.slice/crio-3fecbc442422605708f904bbca2e6e15a611af5f62abe74bbb9b35c9eb34da41 WatchSource:0}: Error finding container 3fecbc442422605708f904bbca2e6e15a611af5f62abe74bbb9b35c9eb34da41: Status 404 returned error can't find the container with id 3fecbc442422605708f904bbca2e6e15a611af5f62abe74bbb9b35c9eb34da41
Oct 30 22:49:48.194680 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:49:48.194646    4412 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-37d062d1dd/isvc-lightgbm-runtime-predictor-5c98bb5cc5-m85nn" event={"ID":"b5645218-816f-402f-92be-184fafc6e4de","Type":"ContainerStarted","Data":"0cdc9a70ff181c1836b83f7fd30f1a2de3d8acabc2f91032a2836b1c5e376163"}
Oct 30 22:49:48.195059 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:49:48.194703    4412 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-37d062d1dd/isvc-lightgbm-runtime-predictor-5c98bb5cc5-m85nn" event={"ID":"b5645218-816f-402f-92be-184fafc6e4de","Type":"ContainerStarted","Data":"3fecbc442422605708f904bbca2e6e15a611af5f62abe74bbb9b35c9eb34da41"}
Oct 30 22:49:52.214022 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:49:52.213986    4412 generic.go:356] "Generic (PLEG): container finished" podID="0be79600-f0e1-4f26-bf22-dec5d2079875" containerID="79d8e19e48556976e586e9cc4e5e6e47fdc99d3f284cf3a767c934cbe8fcc8d2" exitCode=0
Oct 30 22:49:52.214478 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:49:52.214054    4412 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-7476d02544/isvc-paddle-predictor-74cbfb4bcb-d9zph" event={"ID":"0be79600-f0e1-4f26-bf22-dec5d2079875","Type":"ContainerDied","Data":"79d8e19e48556976e586e9cc4e5e6e47fdc99d3f284cf3a767c934cbe8fcc8d2"}
Oct 30 22:49:52.215607 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:49:52.215590    4412 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 22:49:52.215794 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:49:52.215775    4412 generic.go:356] "Generic (PLEG): container finished" podID="b5645218-816f-402f-92be-184fafc6e4de" containerID="0cdc9a70ff181c1836b83f7fd30f1a2de3d8acabc2f91032a2836b1c5e376163" exitCode=0
Oct 30 22:49:52.215849 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:49:52.215836    4412 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-37d062d1dd/isvc-lightgbm-runtime-predictor-5c98bb5cc5-m85nn" event={"ID":"b5645218-816f-402f-92be-184fafc6e4de","Type":"ContainerDied","Data":"0cdc9a70ff181c1836b83f7fd30f1a2de3d8acabc2f91032a2836b1c5e376163"}
Oct 30 22:50:01.259900 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:50:01.259862    4412 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-37d062d1dd/isvc-lightgbm-runtime-predictor-5c98bb5cc5-m85nn" event={"ID":"b5645218-816f-402f-92be-184fafc6e4de","Type":"ContainerStarted","Data":"575e275b949accfec6ab2f886925a5fce01a6160591dc779bc6a94bc70040714"}
Oct 30 22:50:01.260475 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:50:01.259943    4412 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-37d062d1dd/isvc-lightgbm-runtime-predictor-5c98bb5cc5-m85nn" event={"ID":"b5645218-816f-402f-92be-184fafc6e4de","Type":"ContainerStarted","Data":"e9b2df62430194927a60f3e5bb91766c2a50aeaa6064780c90e789f426e17409"}
Oct 30 22:50:01.260475 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:50:01.260209    4412 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw0-37d062d1dd/isvc-lightgbm-runtime-predictor-5c98bb5cc5-m85nn"
Oct 30 22:50:01.431468 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:50:01.431397    4412 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-gw0-37d062d1dd/isvc-lightgbm-runtime-predictor-5c98bb5cc5-m85nn" podStartSLOduration=2.878267003 podStartE2EDuration="15.431382683s" totalImagesPullingTime="8.455010841s" totalInitContainerRuntime="4.098104839s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:49:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:50:01.430076039 +0000 UTC m=+3421.934747992" watchObservedRunningTime="2026-10-30 22:50:01.431382683 +0000 UTC m=+3421.936054698"
Oct 30 22:50:02.263927 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:50:02.263857    4412 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw0-37d062d1dd/isvc-lightgbm-runtime-predictor-5c98bb5cc5-m85nn"
Oct 30 22:50:02.265647 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:50:02.265616    4412 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-37d062d1dd/isvc-lightgbm-runtime-predictor-5c98bb5cc5-m85nn" podUID="b5645218-816f-402f-92be-184fafc6e4de" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.35:8080: connect: connection refused"
Oct 30 22:50:03.267812 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:50:03.267768    4412 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-37d062d1dd/isvc-lightgbm-runtime-predictor-5c98bb5cc5-m85nn" podUID="b5645218-816f-402f-92be-184fafc6e4de" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.35:8080: connect: connection refused"
Oct 30 22:50:04.454362 ip-10-0-131-202 kubenswrapper[4412]: E1030 22:50:04.454327    4412 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bkhmz" containerName="extractor"
Oct 30 22:50:07.284645 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:50:07.284612    4412 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-7476d02544/isvc-paddle-predictor-74cbfb4bcb-d9zph" event={"ID":"0be79600-f0e1-4f26-bf22-dec5d2079875","Type":"ContainerStarted","Data":"d65a47d29e297813a72b3d7a3ccffc32b26284cc38e5a996c6fbe5a9a6c5791b"}
Oct 30 22:50:07.285045 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:50:07.284684    4412 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-7476d02544/isvc-paddle-predictor-74cbfb4bcb-d9zph" event={"ID":"0be79600-f0e1-4f26-bf22-dec5d2079875","Type":"ContainerStarted","Data":"dcd8f694076f447aad9c8619273b535be7b6cefdcf8884e214547c75e9a76a5d"}
Oct 30 22:50:07.285045 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:50:07.284896    4412 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw1-7476d02544/isvc-paddle-predictor-74cbfb4bcb-d9zph"
Oct 30 22:50:07.488671 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:50:07.488620    4412 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-gw1-7476d02544/isvc-paddle-predictor-74cbfb4bcb-d9zph" podStartSLOduration=2.04721636 podStartE2EDuration="22.488605537s" totalImagesPullingTime="14.418553635s" totalInitContainerRuntime="6.022835542s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:49:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:50:07.456584552 +0000 UTC m=+3427.961256504" watchObservedRunningTime="2026-10-30 22:50:07.488605537 +0000 UTC m=+3427.993277487"
Oct 30 22:50:08.272713 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:50:08.272684    4412 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw0-37d062d1dd/isvc-lightgbm-runtime-predictor-5c98bb5cc5-m85nn"
Oct 30 22:50:08.273195 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:50:08.273163    4412 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-37d062d1dd/isvc-lightgbm-runtime-predictor-5c98bb5cc5-m85nn" podUID="b5645218-816f-402f-92be-184fafc6e4de" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.35:8080: connect: connection refused"
Oct 30 22:50:08.288192 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:50:08.288169    4412 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw1-7476d02544/isvc-paddle-predictor-74cbfb4bcb-d9zph"
Oct 30 22:50:08.289460 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:50:08.289436    4412 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-7476d02544/isvc-paddle-predictor-74cbfb4bcb-d9zph" podUID="0be79600-f0e1-4f26-bf22-dec5d2079875" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.34:8080: connect: connection refused"
Oct 30 22:50:09.292465 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:50:09.292431    4412 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-7476d02544/isvc-paddle-predictor-74cbfb4bcb-d9zph" podUID="0be79600-f0e1-4f26-bf22-dec5d2079875" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.34:8080: connect: connection refused"
Oct 30 22:50:14.297099 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:50:14.297067    4412 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw1-7476d02544/isvc-paddle-predictor-74cbfb4bcb-d9zph"
Oct 30 22:50:14.297679 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:50:14.297650    4412 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-7476d02544/isvc-paddle-predictor-74cbfb4bcb-d9zph" podUID="0be79600-f0e1-4f26-bf22-dec5d2079875" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.34:8080: connect: connection refused"
Oct 30 22:50:18.273431 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:50:18.273395    4412 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-37d062d1dd/isvc-lightgbm-runtime-predictor-5c98bb5cc5-m85nn" podUID="b5645218-816f-402f-92be-184fafc6e4de" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.35:8080: connect: connection refused"
Oct 30 22:50:24.298309 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:50:24.298269    4412 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-7476d02544/isvc-paddle-predictor-74cbfb4bcb-d9zph" podUID="0be79600-f0e1-4f26-bf22-dec5d2079875" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.34:8080: connect: connection refused"
Oct 30 22:50:28.273628 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:50:28.273586    4412 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-37d062d1dd/isvc-lightgbm-runtime-predictor-5c98bb5cc5-m85nn" podUID="b5645218-816f-402f-92be-184fafc6e4de" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.35:8080: connect: connection refused"
Oct 30 22:50:34.298234 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:50:34.298198    4412 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-7476d02544/isvc-paddle-predictor-74cbfb4bcb-d9zph" podUID="0be79600-f0e1-4f26-bf22-dec5d2079875" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.34:8080: connect: connection refused"
Oct 30 22:50:34.451425 ip-10-0-131-202 kubenswrapper[4412]: E1030 22:50:34.451381    4412 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bkhmz" containerName="extractor"
Oct 30 22:50:38.273169 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:50:38.273114    4412 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-37d062d1dd/isvc-lightgbm-runtime-predictor-5c98bb5cc5-m85nn" podUID="b5645218-816f-402f-92be-184fafc6e4de" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.35:8080: connect: connection refused"
Oct 30 22:50:44.297824 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:50:44.297741    4412 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-7476d02544/isvc-paddle-predictor-74cbfb4bcb-d9zph" podUID="0be79600-f0e1-4f26-bf22-dec5d2079875" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.34:8080: connect: connection refused"
Oct 30 22:50:48.273820 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:50:48.273783    4412 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-37d062d1dd/isvc-lightgbm-runtime-predictor-5c98bb5cc5-m85nn" podUID="b5645218-816f-402f-92be-184fafc6e4de" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.35:8080: connect: connection refused"
Oct 30 22:50:54.298087 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:50:54.298060    4412 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw1-7476d02544/isvc-paddle-predictor-74cbfb4bcb-d9zph"
Oct 30 22:50:58.273254 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:50:58.273220    4412 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-37d062d1dd/isvc-lightgbm-runtime-predictor-5c98bb5cc5-m85nn" podUID="b5645218-816f-402f-92be-184fafc6e4de" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.35:8080: connect: connection refused"
Oct 30 22:51:04.449785 ip-10-0-131-202 kubenswrapper[4412]: E1030 22:51:04.449743    4412 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bkhmz" containerName="extractor"
Oct 30 22:51:06.964649 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:51:06.964608    4412 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw1-7476d02544/isvc-paddle-predictor-74cbfb4bcb-d9zph"]
Oct 30 22:51:06.965185 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:51:06.964842    4412 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw1-7476d02544/isvc-paddle-predictor-74cbfb4bcb-d9zph" podUID="0be79600-f0e1-4f26-bf22-dec5d2079875" containerName="kserve-container" containerID="cri-o://dcd8f694076f447aad9c8619273b535be7b6cefdcf8884e214547c75e9a76a5d" gracePeriod=30
Oct 30 22:51:06.965185 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:51:06.964897    4412 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw1-7476d02544/isvc-paddle-predictor-74cbfb4bcb-d9zph" podUID="0be79600-f0e1-4f26-bf22-dec5d2079875" containerName="kube-rbac-proxy" containerID="cri-o://d65a47d29e297813a72b3d7a3ccffc32b26284cc38e5a996c6fbe5a9a6c5791b" gracePeriod=30
Oct 30 22:51:07.507971 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:51:07.507935    4412 generic.go:356] "Generic (PLEG): container finished" podID="0be79600-f0e1-4f26-bf22-dec5d2079875" containerID="d65a47d29e297813a72b3d7a3ccffc32b26284cc38e5a996c6fbe5a9a6c5791b" exitCode=0
Oct 30 22:51:07.508178 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:51:07.508013    4412 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-7476d02544/isvc-paddle-predictor-74cbfb4bcb-d9zph" event={"ID":"0be79600-f0e1-4f26-bf22-dec5d2079875","Type":"ContainerDied","Data":"d65a47d29e297813a72b3d7a3ccffc32b26284cc38e5a996c6fbe5a9a6c5791b"}
Oct 30 22:51:08.274044 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:51:08.274007    4412 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-37d062d1dd/isvc-lightgbm-runtime-predictor-5c98bb5cc5-m85nn" podUID="b5645218-816f-402f-92be-184fafc6e4de" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.35:8080: connect: connection refused"
Oct 30 22:51:09.293292 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:51:09.293250    4412 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-7476d02544/isvc-paddle-predictor-74cbfb4bcb-d9zph" podUID="0be79600-f0e1-4f26-bf22-dec5d2079875" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.34:8643/healthz\": dial tcp 10.133.0.34:8643: connect: connection refused"
Oct 30 22:51:10.103077 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:51:10.103053    4412 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-7476d02544/isvc-paddle-predictor-74cbfb4bcb-d9zph"
Oct 30 22:51:10.245595 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:51:10.245560    4412 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0be79600-f0e1-4f26-bf22-dec5d2079875-cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/0be79600-f0e1-4f26-bf22-dec5d2079875-cabundle-cert\") pod \"0be79600-f0e1-4f26-bf22-dec5d2079875\" (UID: \"0be79600-f0e1-4f26-bf22-dec5d2079875\") "
Oct 30 22:51:10.245595 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:51:10.245602    4412 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0be79600-f0e1-4f26-bf22-dec5d2079875-kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/0be79600-f0e1-4f26-bf22-dec5d2079875-kube-rbac-proxy-sar-config\") pod \"0be79600-f0e1-4f26-bf22-dec5d2079875\" (UID: \"0be79600-f0e1-4f26-bf22-dec5d2079875\") "
Oct 30 22:51:10.245872 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:51:10.245635    4412 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0be79600-f0e1-4f26-bf22-dec5d2079875-kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/0be79600-f0e1-4f26-bf22-dec5d2079875-kserve-provision-location\") pod \"0be79600-f0e1-4f26-bf22-dec5d2079875\" (UID: \"0be79600-f0e1-4f26-bf22-dec5d2079875\") "
Oct 30 22:51:10.245872 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:51:10.245752    4412 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0be79600-f0e1-4f26-bf22-dec5d2079875-kube-api-access-8dhn4\" (UniqueName: \"kubernetes.io/projected/0be79600-f0e1-4f26-bf22-dec5d2079875-kube-api-access-8dhn4\") pod \"0be79600-f0e1-4f26-bf22-dec5d2079875\" (UID: \"0be79600-f0e1-4f26-bf22-dec5d2079875\") "
Oct 30 22:51:10.245872 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:51:10.245819    4412 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/0be79600-f0e1-4f26-bf22-dec5d2079875-proxy-tls\" (UniqueName: \"kubernetes.io/secret/0be79600-f0e1-4f26-bf22-dec5d2079875-proxy-tls\") pod \"0be79600-f0e1-4f26-bf22-dec5d2079875\" (UID: \"0be79600-f0e1-4f26-bf22-dec5d2079875\") "
Oct 30 22:51:10.246057 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:51:10.246025    4412 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0be79600-f0e1-4f26-bf22-dec5d2079875-kube-rbac-proxy-sar-config" pod "0be79600-f0e1-4f26-bf22-dec5d2079875" (UID: "0be79600-f0e1-4f26-bf22-dec5d2079875"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:51:10.246139 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:51:10.246039    4412 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0be79600-f0e1-4f26-bf22-dec5d2079875-cabundle-cert" pod "0be79600-f0e1-4f26-bf22-dec5d2079875" (UID: "0be79600-f0e1-4f26-bf22-dec5d2079875"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:51:10.248019 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:51:10.247993    4412 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0be79600-f0e1-4f26-bf22-dec5d2079875-proxy-tls" pod "0be79600-f0e1-4f26-bf22-dec5d2079875" (UID: "0be79600-f0e1-4f26-bf22-dec5d2079875"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:51:10.248332 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:51:10.248301    4412 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0be79600-f0e1-4f26-bf22-dec5d2079875-kube-api-access-8dhn4" pod "0be79600-f0e1-4f26-bf22-dec5d2079875" (UID: "0be79600-f0e1-4f26-bf22-dec5d2079875"). InnerVolumeSpecName "kube-api-access-8dhn4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:51:10.255624 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:51:10.255601    4412 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0be79600-f0e1-4f26-bf22-dec5d2079875-kserve-provision-location" pod "0be79600-f0e1-4f26-bf22-dec5d2079875" (UID: "0be79600-f0e1-4f26-bf22-dec5d2079875"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:51:10.347379 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:51:10.347343    4412 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8dhn4\" (UniqueName: \"kubernetes.io/projected/0be79600-f0e1-4f26-bf22-dec5d2079875-kube-api-access-8dhn4\") on node \"ip-10-0-131-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:51:10.347379 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:51:10.347370    4412 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/0be79600-f0e1-4f26-bf22-dec5d2079875-proxy-tls\") on node \"ip-10-0-131-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:51:10.347379 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:51:10.347380    4412 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/0be79600-f0e1-4f26-bf22-dec5d2079875-cabundle-cert\") on node \"ip-10-0-131-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:51:10.347379 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:51:10.347390    4412 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/0be79600-f0e1-4f26-bf22-dec5d2079875-kube-rbac-proxy-sar-config\") on node \"ip-10-0-131-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:51:10.347851 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:51:10.347398    4412 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/0be79600-f0e1-4f26-bf22-dec5d2079875-kserve-provision-location\") on node \"ip-10-0-131-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:51:10.521655 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:51:10.521571    4412 generic.go:356] "Generic (PLEG): container finished" podID="0be79600-f0e1-4f26-bf22-dec5d2079875" containerID="dcd8f694076f447aad9c8619273b535be7b6cefdcf8884e214547c75e9a76a5d" exitCode=0
Oct 30 22:51:10.521655 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:51:10.521606    4412 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-7476d02544/isvc-paddle-predictor-74cbfb4bcb-d9zph" event={"ID":"0be79600-f0e1-4f26-bf22-dec5d2079875","Type":"ContainerDied","Data":"dcd8f694076f447aad9c8619273b535be7b6cefdcf8884e214547c75e9a76a5d"}
Oct 30 22:51:10.521655 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:51:10.521633    4412 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-7476d02544/isvc-paddle-predictor-74cbfb4bcb-d9zph"
Oct 30 22:51:10.521655 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:51:10.521654    4412 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-7476d02544/isvc-paddle-predictor-74cbfb4bcb-d9zph" event={"ID":"0be79600-f0e1-4f26-bf22-dec5d2079875","Type":"ContainerDied","Data":"daa74435d2df5e27fb3550c70d7438d31514f7095bce289f7c89c1686abc9ede"}
Oct 30 22:51:10.521943 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:51:10.521680    4412 scope.go:122] "RemoveContainer" containerID="d65a47d29e297813a72b3d7a3ccffc32b26284cc38e5a996c6fbe5a9a6c5791b"
Oct 30 22:51:10.530516 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:51:10.530496    4412 scope.go:122] "RemoveContainer" containerID="dcd8f694076f447aad9c8619273b535be7b6cefdcf8884e214547c75e9a76a5d"
Oct 30 22:51:10.537956 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:51:10.537933    4412 scope.go:122] "RemoveContainer" containerID="79d8e19e48556976e586e9cc4e5e6e47fdc99d3f284cf3a767c934cbe8fcc8d2"
Oct 30 22:51:10.546479 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:51:10.546460    4412 scope.go:122] "RemoveContainer" containerID="d65a47d29e297813a72b3d7a3ccffc32b26284cc38e5a996c6fbe5a9a6c5791b"
Oct 30 22:51:10.546737 ip-10-0-131-202 kubenswrapper[4412]: E1030 22:51:10.546719    4412 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d65a47d29e297813a72b3d7a3ccffc32b26284cc38e5a996c6fbe5a9a6c5791b\": container with ID starting with d65a47d29e297813a72b3d7a3ccffc32b26284cc38e5a996c6fbe5a9a6c5791b not found: ID does not exist" containerID="d65a47d29e297813a72b3d7a3ccffc32b26284cc38e5a996c6fbe5a9a6c5791b"
Oct 30 22:51:10.546802 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:51:10.546744    4412 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d65a47d29e297813a72b3d7a3ccffc32b26284cc38e5a996c6fbe5a9a6c5791b"} err="failed to get container status \"d65a47d29e297813a72b3d7a3ccffc32b26284cc38e5a996c6fbe5a9a6c5791b\": rpc error: code = NotFound desc = could not find container \"d65a47d29e297813a72b3d7a3ccffc32b26284cc38e5a996c6fbe5a9a6c5791b\": container with ID starting with d65a47d29e297813a72b3d7a3ccffc32b26284cc38e5a996c6fbe5a9a6c5791b not found: ID does not exist"
Oct 30 22:51:10.546802 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:51:10.546761    4412 scope.go:122] "RemoveContainer" containerID="dcd8f694076f447aad9c8619273b535be7b6cefdcf8884e214547c75e9a76a5d"
Oct 30 22:51:10.547002 ip-10-0-131-202 kubenswrapper[4412]: E1030 22:51:10.546986    4412 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dcd8f694076f447aad9c8619273b535be7b6cefdcf8884e214547c75e9a76a5d\": container with ID starting with dcd8f694076f447aad9c8619273b535be7b6cefdcf8884e214547c75e9a76a5d not found: ID does not exist" containerID="dcd8f694076f447aad9c8619273b535be7b6cefdcf8884e214547c75e9a76a5d"
Oct 30 22:51:10.547049 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:51:10.547007    4412 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dcd8f694076f447aad9c8619273b535be7b6cefdcf8884e214547c75e9a76a5d"} err="failed to get container status \"dcd8f694076f447aad9c8619273b535be7b6cefdcf8884e214547c75e9a76a5d\": rpc error: code = NotFound desc = could not find container \"dcd8f694076f447aad9c8619273b535be7b6cefdcf8884e214547c75e9a76a5d\": container with ID starting with dcd8f694076f447aad9c8619273b535be7b6cefdcf8884e214547c75e9a76a5d not found: ID does not exist"
Oct 30 22:51:10.547049 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:51:10.547020    4412 scope.go:122] "RemoveContainer" containerID="79d8e19e48556976e586e9cc4e5e6e47fdc99d3f284cf3a767c934cbe8fcc8d2"
Oct 30 22:51:10.547353 ip-10-0-131-202 kubenswrapper[4412]: E1030 22:51:10.547335    4412 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"79d8e19e48556976e586e9cc4e5e6e47fdc99d3f284cf3a767c934cbe8fcc8d2\": container with ID starting with 79d8e19e48556976e586e9cc4e5e6e47fdc99d3f284cf3a767c934cbe8fcc8d2 not found: ID does not exist" containerID="79d8e19e48556976e586e9cc4e5e6e47fdc99d3f284cf3a767c934cbe8fcc8d2"
Oct 30 22:51:10.547403 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:51:10.547360    4412 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"79d8e19e48556976e586e9cc4e5e6e47fdc99d3f284cf3a767c934cbe8fcc8d2"} err="failed to get container status \"79d8e19e48556976e586e9cc4e5e6e47fdc99d3f284cf3a767c934cbe8fcc8d2\": rpc error: code = NotFound desc = could not find container \"79d8e19e48556976e586e9cc4e5e6e47fdc99d3f284cf3a767c934cbe8fcc8d2\": container with ID starting with 79d8e19e48556976e586e9cc4e5e6e47fdc99d3f284cf3a767c934cbe8fcc8d2 not found: ID does not exist"
Oct 30 22:51:10.687133 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:51:10.687087    4412 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw1-7476d02544/isvc-paddle-predictor-74cbfb4bcb-d9zph"]
Oct 30 22:51:10.737525 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:51:10.737491    4412 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-gw1-7476d02544/isvc-paddle-predictor-74cbfb4bcb-d9zph"]
Oct 30 22:51:11.625576 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:51:11.625547    4412 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0be79600-f0e1-4f26-bf22-dec5d2079875" path="/var/lib/kubelet/pods/0be79600-f0e1-4f26-bf22-dec5d2079875/volumes"
Oct 30 22:51:13.991408 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:51:13.991374    4412 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-gw1-7476d02544/isvc-paddle-runtime-predictor-57968fb656-2z7dr"]
Oct 30 22:51:13.991838 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:51:13.991666    4412 cpu_manager.go:414] "containerMap: removing container" podUID="0be79600-f0e1-4f26-bf22-dec5d2079875" containerName="kube-rbac-proxy"
Oct 30 22:51:13.991838 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:51:13.991676    4412 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0be79600-f0e1-4f26-bf22-dec5d2079875" containerName="kube-rbac-proxy"
Oct 30 22:51:13.991838 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:51:13.991691    4412 cpu_manager.go:414] "containerMap: removing container" podUID="0be79600-f0e1-4f26-bf22-dec5d2079875" containerName="kserve-container"
Oct 30 22:51:13.991838 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:51:13.991696    4412 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0be79600-f0e1-4f26-bf22-dec5d2079875" containerName="kserve-container"
Oct 30 22:51:13.991838 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:51:13.991715    4412 cpu_manager.go:414] "containerMap: removing container" podUID="0be79600-f0e1-4f26-bf22-dec5d2079875" containerName="storage-initializer"
Oct 30 22:51:13.991838 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:51:13.991720    4412 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0be79600-f0e1-4f26-bf22-dec5d2079875" containerName="storage-initializer"
Oct 30 22:51:13.991838 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:51:13.991764    4412 memory_manager.go:365] "RemoveStaleState removing state" podUID="0be79600-f0e1-4f26-bf22-dec5d2079875" containerName="kube-rbac-proxy"
Oct 30 22:51:13.991838 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:51:13.991773    4412 memory_manager.go:365] "RemoveStaleState removing state" podUID="0be79600-f0e1-4f26-bf22-dec5d2079875" containerName="kserve-container"
Oct 30 22:51:13.994438 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:51:13.994421    4412 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-7476d02544/isvc-paddle-runtime-predictor-57968fb656-2z7dr"
Oct 30 22:51:14.097949 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:51:14.097887    4412 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw1-7476d02544\"/\"seaweedfs-s3-creds\""
Oct 30 22:51:14.097949 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:51:14.097948    4412 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw1-7476d02544\"/\"kube-root-ca.crt\""
Oct 30 22:51:14.098216 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:51:14.097976    4412 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw1-7476d02544\"/\"odh-kserve-custom-ca-bundle\""
Oct 30 22:51:14.098216 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:51:14.098002    4412 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw1-7476d02544\"/\"isvc-paddle-runtime-kube-rbac-proxy-sar-config\""
Oct 30 22:51:14.098216 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:51:14.098030    4412 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw1-7476d02544\"/\"openshift-service-ca.crt\""
Oct 30 22:51:14.098216 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:51:14.098102    4412 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw1-7476d02544\"/\"default-dockercfg-cxfkn\""
Oct 30 22:51:14.098376 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:51:14.098246    4412 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw1-7476d02544\"/\"isvc-paddle-runtime-predictor-serving-cert\""
Oct 30 22:51:14.099723 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:51:14.099697    4412 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw1-7476d02544/isvc-paddle-runtime-predictor-57968fb656-2z7dr"]
Oct 30 22:51:14.177588 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:51:14.177555    4412 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/7797d891-fae0-4c51-82aa-df9f03a337b1-kube-rbac-proxy-sar-config\") pod \"isvc-paddle-runtime-predictor-57968fb656-2z7dr\" (UID: \"7797d891-fae0-4c51-82aa-df9f03a337b1\") " pod="e2e-gw1-7476d02544/isvc-paddle-runtime-predictor-57968fb656-2z7dr"
Oct 30 22:51:14.177588 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:51:14.177589    4412 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m24bw\" (UniqueName: \"kubernetes.io/projected/7797d891-fae0-4c51-82aa-df9f03a337b1-kube-api-access-m24bw\") pod \"isvc-paddle-runtime-predictor-57968fb656-2z7dr\" (UID: \"7797d891-fae0-4c51-82aa-df9f03a337b1\") " pod="e2e-gw1-7476d02544/isvc-paddle-runtime-predictor-57968fb656-2z7dr"
Oct 30 22:51:14.177854 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:51:14.177610    4412 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/7797d891-fae0-4c51-82aa-df9f03a337b1-proxy-tls\") pod \"isvc-paddle-runtime-predictor-57968fb656-2z7dr\" (UID: \"7797d891-fae0-4c51-82aa-df9f03a337b1\") " pod="e2e-gw1-7476d02544/isvc-paddle-runtime-predictor-57968fb656-2z7dr"
Oct 30 22:51:14.177854 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:51:14.177796    4412 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/7797d891-fae0-4c51-82aa-df9f03a337b1-kserve-provision-location\") pod \"isvc-paddle-runtime-predictor-57968fb656-2z7dr\" (UID: \"7797d891-fae0-4c51-82aa-df9f03a337b1\") " pod="e2e-gw1-7476d02544/isvc-paddle-runtime-predictor-57968fb656-2z7dr"
Oct 30 22:51:14.177854 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:51:14.177838    4412 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/7797d891-fae0-4c51-82aa-df9f03a337b1-cabundle-cert\") pod \"isvc-paddle-runtime-predictor-57968fb656-2z7dr\" (UID: \"7797d891-fae0-4c51-82aa-df9f03a337b1\") " pod="e2e-gw1-7476d02544/isvc-paddle-runtime-predictor-57968fb656-2z7dr"
Oct 30 22:51:14.279091 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:51:14.278986    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m24bw\" (UniqueName: \"kubernetes.io/projected/7797d891-fae0-4c51-82aa-df9f03a337b1-kube-api-access-m24bw\") pod \"isvc-paddle-runtime-predictor-57968fb656-2z7dr\" (UID: \"7797d891-fae0-4c51-82aa-df9f03a337b1\") " pod="e2e-gw1-7476d02544/isvc-paddle-runtime-predictor-57968fb656-2z7dr"
Oct 30 22:51:14.279091 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:51:14.279038    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/7797d891-fae0-4c51-82aa-df9f03a337b1-proxy-tls\") pod \"isvc-paddle-runtime-predictor-57968fb656-2z7dr\" (UID: \"7797d891-fae0-4c51-82aa-df9f03a337b1\") " pod="e2e-gw1-7476d02544/isvc-paddle-runtime-predictor-57968fb656-2z7dr"
Oct 30 22:51:14.279091 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:51:14.279095    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/7797d891-fae0-4c51-82aa-df9f03a337b1-kserve-provision-location\") pod \"isvc-paddle-runtime-predictor-57968fb656-2z7dr\" (UID: \"7797d891-fae0-4c51-82aa-df9f03a337b1\") " pod="e2e-gw1-7476d02544/isvc-paddle-runtime-predictor-57968fb656-2z7dr"
Oct 30 22:51:14.279445 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:51:14.279113    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/7797d891-fae0-4c51-82aa-df9f03a337b1-cabundle-cert\") pod \"isvc-paddle-runtime-predictor-57968fb656-2z7dr\" (UID: \"7797d891-fae0-4c51-82aa-df9f03a337b1\") " pod="e2e-gw1-7476d02544/isvc-paddle-runtime-predictor-57968fb656-2z7dr"
Oct 30 22:51:14.279445 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:51:14.279163    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/7797d891-fae0-4c51-82aa-df9f03a337b1-kube-rbac-proxy-sar-config\") pod \"isvc-paddle-runtime-predictor-57968fb656-2z7dr\" (UID: \"7797d891-fae0-4c51-82aa-df9f03a337b1\") " pod="e2e-gw1-7476d02544/isvc-paddle-runtime-predictor-57968fb656-2z7dr"
Oct 30 22:51:14.279587 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:51:14.279563    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/7797d891-fae0-4c51-82aa-df9f03a337b1-kserve-provision-location\") pod \"isvc-paddle-runtime-predictor-57968fb656-2z7dr\" (UID: \"7797d891-fae0-4c51-82aa-df9f03a337b1\") " pod="e2e-gw1-7476d02544/isvc-paddle-runtime-predictor-57968fb656-2z7dr"
Oct 30 22:51:14.279850 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:51:14.279829    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/7797d891-fae0-4c51-82aa-df9f03a337b1-kube-rbac-proxy-sar-config\") pod \"isvc-paddle-runtime-predictor-57968fb656-2z7dr\" (UID: \"7797d891-fae0-4c51-82aa-df9f03a337b1\") " pod="e2e-gw1-7476d02544/isvc-paddle-runtime-predictor-57968fb656-2z7dr"
Oct 30 22:51:14.279941 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:51:14.279912    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/7797d891-fae0-4c51-82aa-df9f03a337b1-cabundle-cert\") pod \"isvc-paddle-runtime-predictor-57968fb656-2z7dr\" (UID: \"7797d891-fae0-4c51-82aa-df9f03a337b1\") " pod="e2e-gw1-7476d02544/isvc-paddle-runtime-predictor-57968fb656-2z7dr"
Oct 30 22:51:14.281695 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:51:14.281672    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/7797d891-fae0-4c51-82aa-df9f03a337b1-proxy-tls\") pod \"isvc-paddle-runtime-predictor-57968fb656-2z7dr\" (UID: \"7797d891-fae0-4c51-82aa-df9f03a337b1\") " pod="e2e-gw1-7476d02544/isvc-paddle-runtime-predictor-57968fb656-2z7dr"
Oct 30 22:51:14.337441 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:51:14.337409    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m24bw\" (UniqueName: \"kubernetes.io/projected/7797d891-fae0-4c51-82aa-df9f03a337b1-kube-api-access-m24bw\") pod \"isvc-paddle-runtime-predictor-57968fb656-2z7dr\" (UID: \"7797d891-fae0-4c51-82aa-df9f03a337b1\") " pod="e2e-gw1-7476d02544/isvc-paddle-runtime-predictor-57968fb656-2z7dr"
Oct 30 22:51:14.607243 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:51:14.607167    4412 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-7476d02544/isvc-paddle-runtime-predictor-57968fb656-2z7dr"
Oct 30 22:51:15.027242 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:51:15.027190    4412 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw1-7476d02544/isvc-paddle-runtime-predictor-57968fb656-2z7dr"]
Oct 30 22:51:15.029577 ip-10-0-131-202 kubenswrapper[4412]: W1030 22:51:15.029549    4412 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7797d891_fae0_4c51_82aa_df9f03a337b1.slice/crio-eeadc94cfb58cd45f6be19edc6363e4ea793ce6b952a8c2c0688bd1d05688358 WatchSource:0}: Error finding container eeadc94cfb58cd45f6be19edc6363e4ea793ce6b952a8c2c0688bd1d05688358: Status 404 returned error can't find the container with id eeadc94cfb58cd45f6be19edc6363e4ea793ce6b952a8c2c0688bd1d05688358
Oct 30 22:51:15.548181 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:51:15.548147    4412 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-7476d02544/isvc-paddle-runtime-predictor-57968fb656-2z7dr" event={"ID":"7797d891-fae0-4c51-82aa-df9f03a337b1","Type":"ContainerStarted","Data":"c61f15776e76125670b9560e36147c3c5331767b3f3e3b89e85c8d5c69d8d130"}
Oct 30 22:51:15.548366 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:51:15.548202    4412 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-7476d02544/isvc-paddle-runtime-predictor-57968fb656-2z7dr" event={"ID":"7797d891-fae0-4c51-82aa-df9f03a337b1","Type":"ContainerStarted","Data":"eeadc94cfb58cd45f6be19edc6363e4ea793ce6b952a8c2c0688bd1d05688358"}
Oct 30 22:51:18.273802 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:51:18.273766    4412 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-37d062d1dd/isvc-lightgbm-runtime-predictor-5c98bb5cc5-m85nn" podUID="b5645218-816f-402f-92be-184fafc6e4de" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.35:8080: connect: connection refused"
Oct 30 22:51:20.568680 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:51:20.568641    4412 generic.go:356] "Generic (PLEG): container finished" podID="7797d891-fae0-4c51-82aa-df9f03a337b1" containerID="c61f15776e76125670b9560e36147c3c5331767b3f3e3b89e85c8d5c69d8d130" exitCode=0
Oct 30 22:51:20.569088 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:51:20.568708    4412 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-7476d02544/isvc-paddle-runtime-predictor-57968fb656-2z7dr" event={"ID":"7797d891-fae0-4c51-82aa-df9f03a337b1","Type":"ContainerDied","Data":"c61f15776e76125670b9560e36147c3c5331767b3f3e3b89e85c8d5c69d8d130"}
Oct 30 22:51:21.574646 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:51:21.574612    4412 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-7476d02544/isvc-paddle-runtime-predictor-57968fb656-2z7dr" event={"ID":"7797d891-fae0-4c51-82aa-df9f03a337b1","Type":"ContainerStarted","Data":"870d9bb77427ae0dbd16f600572f545c085360041003b0fcbc60aafc711ee2e9"}
Oct 30 22:51:21.575049 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:51:21.574678    4412 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-7476d02544/isvc-paddle-runtime-predictor-57968fb656-2z7dr" event={"ID":"7797d891-fae0-4c51-82aa-df9f03a337b1","Type":"ContainerStarted","Data":"56a511707da1718542a7f6d2ba2409e68e7f32e580a1cdc3ddc8f9e3e46ef168"}
Oct 30 22:51:21.575049 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:51:21.574888    4412 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw1-7476d02544/isvc-paddle-runtime-predictor-57968fb656-2z7dr"
Oct 30 22:51:21.745555 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:51:21.745483    4412 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-gw1-7476d02544/isvc-paddle-runtime-predictor-57968fb656-2z7dr" podStartSLOduration=3.923675865 podStartE2EDuration="8.74546482s" totalImagesPullingTime="0s" totalInitContainerRuntime="4.821788955s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:51:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:51:21.744017467 +0000 UTC m=+3502.248689397" watchObservedRunningTime="2026-10-30 22:51:21.74546482 +0000 UTC m=+3502.250136772"
Oct 30 22:51:22.578643 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:51:22.578615    4412 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw1-7476d02544/isvc-paddle-runtime-predictor-57968fb656-2z7dr"
Oct 30 22:51:22.579987 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:51:22.579952    4412 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-7476d02544/isvc-paddle-runtime-predictor-57968fb656-2z7dr" podUID="7797d891-fae0-4c51-82aa-df9f03a337b1" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.36:8080: connect: connection refused"
Oct 30 22:51:23.582589 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:51:23.582555    4412 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-7476d02544/isvc-paddle-runtime-predictor-57968fb656-2z7dr" podUID="7797d891-fae0-4c51-82aa-df9f03a337b1" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.36:8080: connect: connection refused"
Oct 30 22:51:28.273268 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:51:28.273230    4412 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-37d062d1dd/isvc-lightgbm-runtime-predictor-5c98bb5cc5-m85nn" podUID="b5645218-816f-402f-92be-184fafc6e4de" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.35:8080: connect: connection refused"
Oct 30 22:51:28.586857 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:51:28.586772    4412 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw1-7476d02544/isvc-paddle-runtime-predictor-57968fb656-2z7dr"
Oct 30 22:51:28.587415 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:51:28.587379    4412 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-7476d02544/isvc-paddle-runtime-predictor-57968fb656-2z7dr" podUID="7797d891-fae0-4c51-82aa-df9f03a337b1" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.36:8080: connect: connection refused"
Oct 30 22:51:34.448826 ip-10-0-131-202 kubenswrapper[4412]: E1030 22:51:34.448784    4412 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bkhmz" containerName="extractor"
Oct 30 22:51:34.622104 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:51:34.622077    4412 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw0-37d062d1dd/isvc-lightgbm-runtime-predictor-5c98bb5cc5-m85nn"
Oct 30 22:51:38.588191 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:51:38.588150    4412 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-7476d02544/isvc-paddle-runtime-predictor-57968fb656-2z7dr" podUID="7797d891-fae0-4c51-82aa-df9f03a337b1" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.36:8080: connect: connection refused"
Oct 30 22:51:48.588343 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:51:48.588299    4412 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-7476d02544/isvc-paddle-runtime-predictor-57968fb656-2z7dr" podUID="7797d891-fae0-4c51-82aa-df9f03a337b1" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.36:8080: connect: connection refused"
Oct 30 22:51:48.688441 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:51:48.688411    4412 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw0-37d062d1dd/isvc-lightgbm-runtime-predictor-5c98bb5cc5-m85nn"]
Oct 30 22:51:48.688696 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:51:48.688651    4412 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw0-37d062d1dd/isvc-lightgbm-runtime-predictor-5c98bb5cc5-m85nn" podUID="b5645218-816f-402f-92be-184fafc6e4de" containerName="kserve-container" containerID="cri-o://e9b2df62430194927a60f3e5bb91766c2a50aeaa6064780c90e789f426e17409" gracePeriod=30
Oct 30 22:51:48.688793 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:51:48.688737    4412 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw0-37d062d1dd/isvc-lightgbm-runtime-predictor-5c98bb5cc5-m85nn" podUID="b5645218-816f-402f-92be-184fafc6e4de" containerName="kube-rbac-proxy" containerID="cri-o://575e275b949accfec6ab2f886925a5fce01a6160591dc779bc6a94bc70040714" gracePeriod=30
Oct 30 22:51:49.681192 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:51:49.681162    4412 generic.go:356] "Generic (PLEG): container finished" podID="b5645218-816f-402f-92be-184fafc6e4de" containerID="575e275b949accfec6ab2f886925a5fce01a6160591dc779bc6a94bc70040714" exitCode=0
Oct 30 22:51:49.681588 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:51:49.681227    4412 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-37d062d1dd/isvc-lightgbm-runtime-predictor-5c98bb5cc5-m85nn" event={"ID":"b5645218-816f-402f-92be-184fafc6e4de","Type":"ContainerDied","Data":"575e275b949accfec6ab2f886925a5fce01a6160591dc779bc6a94bc70040714"}
Oct 30 22:51:53.268465 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:51:53.268417    4412 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-37d062d1dd/isvc-lightgbm-runtime-predictor-5c98bb5cc5-m85nn" podUID="b5645218-816f-402f-92be-184fafc6e4de" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.35:8643/healthz\": dial tcp 10.133.0.35:8643: connect: connection refused"
Oct 30 22:51:54.427567 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:51:54.427542    4412 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-37d062d1dd/isvc-lightgbm-runtime-predictor-5c98bb5cc5-m85nn"
Oct 30 22:51:54.507263 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:51:54.507229    4412 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b5645218-816f-402f-92be-184fafc6e4de-kube-api-access-sww6q\" (UniqueName: \"kubernetes.io/projected/b5645218-816f-402f-92be-184fafc6e4de-kube-api-access-sww6q\") pod \"b5645218-816f-402f-92be-184fafc6e4de\" (UID: \"b5645218-816f-402f-92be-184fafc6e4de\") "
Oct 30 22:51:54.507457 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:51:54.507294    4412 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/b5645218-816f-402f-92be-184fafc6e4de-proxy-tls\" (UniqueName: \"kubernetes.io/secret/b5645218-816f-402f-92be-184fafc6e4de-proxy-tls\") pod \"b5645218-816f-402f-92be-184fafc6e4de\" (UID: \"b5645218-816f-402f-92be-184fafc6e4de\") "
Oct 30 22:51:54.507457 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:51:54.507362    4412 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b5645218-816f-402f-92be-184fafc6e4de-kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/b5645218-816f-402f-92be-184fafc6e4de-kube-rbac-proxy-sar-config\") pod \"b5645218-816f-402f-92be-184fafc6e4de\" (UID: \"b5645218-816f-402f-92be-184fafc6e4de\") "
Oct 30 22:51:54.507457 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:51:54.507386    4412 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b5645218-816f-402f-92be-184fafc6e4de-kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/b5645218-816f-402f-92be-184fafc6e4de-kserve-provision-location\") pod \"b5645218-816f-402f-92be-184fafc6e4de\" (UID: \"b5645218-816f-402f-92be-184fafc6e4de\") "
Oct 30 22:51:54.507457 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:51:54.507418    4412 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b5645218-816f-402f-92be-184fafc6e4de-cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/b5645218-816f-402f-92be-184fafc6e4de-cabundle-cert\") pod \"b5645218-816f-402f-92be-184fafc6e4de\" (UID: \"b5645218-816f-402f-92be-184fafc6e4de\") "
Oct 30 22:51:54.507847 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:51:54.507816    4412 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b5645218-816f-402f-92be-184fafc6e4de-kube-rbac-proxy-sar-config" pod "b5645218-816f-402f-92be-184fafc6e4de" (UID: "b5645218-816f-402f-92be-184fafc6e4de"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:51:54.507847 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:51:54.507834    4412 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b5645218-816f-402f-92be-184fafc6e4de-cabundle-cert" pod "b5645218-816f-402f-92be-184fafc6e4de" (UID: "b5645218-816f-402f-92be-184fafc6e4de"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:51:54.508225 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:51:54.507901    4412 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b5645218-816f-402f-92be-184fafc6e4de-kserve-provision-location" pod "b5645218-816f-402f-92be-184fafc6e4de" (UID: "b5645218-816f-402f-92be-184fafc6e4de"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:51:54.509481 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:51:54.509461    4412 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b5645218-816f-402f-92be-184fafc6e4de-kube-api-access-sww6q" pod "b5645218-816f-402f-92be-184fafc6e4de" (UID: "b5645218-816f-402f-92be-184fafc6e4de"). InnerVolumeSpecName "kube-api-access-sww6q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:51:54.509635 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:51:54.509617    4412 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b5645218-816f-402f-92be-184fafc6e4de-proxy-tls" pod "b5645218-816f-402f-92be-184fafc6e4de" (UID: "b5645218-816f-402f-92be-184fafc6e4de"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:51:54.607974 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:51:54.607900    4412 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/b5645218-816f-402f-92be-184fafc6e4de-proxy-tls\") on node \"ip-10-0-131-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:51:54.607974 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:51:54.607924    4412 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/b5645218-816f-402f-92be-184fafc6e4de-kube-rbac-proxy-sar-config\") on node \"ip-10-0-131-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:51:54.607974 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:51:54.607934    4412 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/b5645218-816f-402f-92be-184fafc6e4de-kserve-provision-location\") on node \"ip-10-0-131-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:51:54.607974 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:51:54.607944    4412 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/b5645218-816f-402f-92be-184fafc6e4de-cabundle-cert\") on node \"ip-10-0-131-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:51:54.607974 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:51:54.607952    4412 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sww6q\" (UniqueName: \"kubernetes.io/projected/b5645218-816f-402f-92be-184fafc6e4de-kube-api-access-sww6q\") on node \"ip-10-0-131-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:51:54.702917 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:51:54.702880    4412 generic.go:356] "Generic (PLEG): container finished" podID="b5645218-816f-402f-92be-184fafc6e4de" containerID="e9b2df62430194927a60f3e5bb91766c2a50aeaa6064780c90e789f426e17409" exitCode=0
Oct 30 22:51:54.703075 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:51:54.702947    4412 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-37d062d1dd/isvc-lightgbm-runtime-predictor-5c98bb5cc5-m85nn"
Oct 30 22:51:54.703075 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:51:54.702947    4412 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-37d062d1dd/isvc-lightgbm-runtime-predictor-5c98bb5cc5-m85nn" event={"ID":"b5645218-816f-402f-92be-184fafc6e4de","Type":"ContainerDied","Data":"e9b2df62430194927a60f3e5bb91766c2a50aeaa6064780c90e789f426e17409"}
Oct 30 22:51:54.703075 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:51:54.703011    4412 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-37d062d1dd/isvc-lightgbm-runtime-predictor-5c98bb5cc5-m85nn" event={"ID":"b5645218-816f-402f-92be-184fafc6e4de","Type":"ContainerDied","Data":"3fecbc442422605708f904bbca2e6e15a611af5f62abe74bbb9b35c9eb34da41"}
Oct 30 22:51:54.703075 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:51:54.703039    4412 scope.go:122] "RemoveContainer" containerID="575e275b949accfec6ab2f886925a5fce01a6160591dc779bc6a94bc70040714"
Oct 30 22:51:54.711836 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:51:54.711813    4412 scope.go:122] "RemoveContainer" containerID="e9b2df62430194927a60f3e5bb91766c2a50aeaa6064780c90e789f426e17409"
Oct 30 22:51:54.719078 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:51:54.719062    4412 scope.go:122] "RemoveContainer" containerID="0cdc9a70ff181c1836b83f7fd30f1a2de3d8acabc2f91032a2836b1c5e376163"
Oct 30 22:51:54.726137 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:51:54.726107    4412 scope.go:122] "RemoveContainer" containerID="575e275b949accfec6ab2f886925a5fce01a6160591dc779bc6a94bc70040714"
Oct 30 22:51:54.726421 ip-10-0-131-202 kubenswrapper[4412]: E1030 22:51:54.726384    4412 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"575e275b949accfec6ab2f886925a5fce01a6160591dc779bc6a94bc70040714\": container with ID starting with 575e275b949accfec6ab2f886925a5fce01a6160591dc779bc6a94bc70040714 not found: ID does not exist" containerID="575e275b949accfec6ab2f886925a5fce01a6160591dc779bc6a94bc70040714"
Oct 30 22:51:54.726495 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:51:54.726432    4412 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"575e275b949accfec6ab2f886925a5fce01a6160591dc779bc6a94bc70040714"} err="failed to get container status \"575e275b949accfec6ab2f886925a5fce01a6160591dc779bc6a94bc70040714\": rpc error: code = NotFound desc = could not find container \"575e275b949accfec6ab2f886925a5fce01a6160591dc779bc6a94bc70040714\": container with ID starting with 575e275b949accfec6ab2f886925a5fce01a6160591dc779bc6a94bc70040714 not found: ID does not exist"
Oct 30 22:51:54.726495 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:51:54.726459    4412 scope.go:122] "RemoveContainer" containerID="e9b2df62430194927a60f3e5bb91766c2a50aeaa6064780c90e789f426e17409"
Oct 30 22:51:54.726711 ip-10-0-131-202 kubenswrapper[4412]: E1030 22:51:54.726694    4412 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e9b2df62430194927a60f3e5bb91766c2a50aeaa6064780c90e789f426e17409\": container with ID starting with e9b2df62430194927a60f3e5bb91766c2a50aeaa6064780c90e789f426e17409 not found: ID does not exist" containerID="e9b2df62430194927a60f3e5bb91766c2a50aeaa6064780c90e789f426e17409"
Oct 30 22:51:54.726754 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:51:54.726717    4412 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e9b2df62430194927a60f3e5bb91766c2a50aeaa6064780c90e789f426e17409"} err="failed to get container status \"e9b2df62430194927a60f3e5bb91766c2a50aeaa6064780c90e789f426e17409\": rpc error: code = NotFound desc = could not find container \"e9b2df62430194927a60f3e5bb91766c2a50aeaa6064780c90e789f426e17409\": container with ID starting with e9b2df62430194927a60f3e5bb91766c2a50aeaa6064780c90e789f426e17409 not found: ID does not exist"
Oct 30 22:51:54.726754 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:51:54.726731    4412 scope.go:122] "RemoveContainer" containerID="0cdc9a70ff181c1836b83f7fd30f1a2de3d8acabc2f91032a2836b1c5e376163"
Oct 30 22:51:54.726970 ip-10-0-131-202 kubenswrapper[4412]: E1030 22:51:54.726953    4412 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0cdc9a70ff181c1836b83f7fd30f1a2de3d8acabc2f91032a2836b1c5e376163\": container with ID starting with 0cdc9a70ff181c1836b83f7fd30f1a2de3d8acabc2f91032a2836b1c5e376163 not found: ID does not exist" containerID="0cdc9a70ff181c1836b83f7fd30f1a2de3d8acabc2f91032a2836b1c5e376163"
Oct 30 22:51:54.727013 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:51:54.726976    4412 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0cdc9a70ff181c1836b83f7fd30f1a2de3d8acabc2f91032a2836b1c5e376163"} err="failed to get container status \"0cdc9a70ff181c1836b83f7fd30f1a2de3d8acabc2f91032a2836b1c5e376163\": rpc error: code = NotFound desc = could not find container \"0cdc9a70ff181c1836b83f7fd30f1a2de3d8acabc2f91032a2836b1c5e376163\": container with ID starting with 0cdc9a70ff181c1836b83f7fd30f1a2de3d8acabc2f91032a2836b1c5e376163 not found: ID does not exist"
Oct 30 22:51:54.868958 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:51:54.868883    4412 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw0-37d062d1dd/isvc-lightgbm-runtime-predictor-5c98bb5cc5-m85nn"]
Oct 30 22:51:54.872847 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:51:54.872820    4412 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-gw0-37d062d1dd/isvc-lightgbm-runtime-predictor-5c98bb5cc5-m85nn"]
Oct 30 22:51:55.626035 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:51:55.625997    4412 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b5645218-816f-402f-92be-184fafc6e4de" path="/var/lib/kubelet/pods/b5645218-816f-402f-92be-184fafc6e4de/volumes"
Oct 30 22:51:58.587333 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:51:58.587301    4412 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-7476d02544/isvc-paddle-runtime-predictor-57968fb656-2z7dr" podUID="7797d891-fae0-4c51-82aa-df9f03a337b1" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.36:8080: connect: connection refused"
Oct 30 22:52:04.448783 ip-10-0-131-202 kubenswrapper[4412]: E1030 22:52:04.448740    4412 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bkhmz" containerName="extractor"
Oct 30 22:52:08.588326 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:52:08.588296    4412 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw1-7476d02544/isvc-paddle-runtime-predictor-57968fb656-2z7dr"
Oct 30 22:52:15.665833 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:52:15.665758    4412 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw1-7476d02544/isvc-paddle-runtime-predictor-57968fb656-2z7dr"]
Oct 30 22:52:15.666361 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:52:15.665984    4412 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw1-7476d02544/isvc-paddle-runtime-predictor-57968fb656-2z7dr" podUID="7797d891-fae0-4c51-82aa-df9f03a337b1" containerName="kserve-container" containerID="cri-o://56a511707da1718542a7f6d2ba2409e68e7f32e580a1cdc3ddc8f9e3e46ef168" gracePeriod=30
Oct 30 22:52:15.666361 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:52:15.666030    4412 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw1-7476d02544/isvc-paddle-runtime-predictor-57968fb656-2z7dr" podUID="7797d891-fae0-4c51-82aa-df9f03a337b1" containerName="kube-rbac-proxy" containerID="cri-o://870d9bb77427ae0dbd16f600572f545c085360041003b0fcbc60aafc711ee2e9" gracePeriod=30
Oct 30 22:52:16.783635 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:52:16.783601    4412 generic.go:356] "Generic (PLEG): container finished" podID="7797d891-fae0-4c51-82aa-df9f03a337b1" containerID="870d9bb77427ae0dbd16f600572f545c085360041003b0fcbc60aafc711ee2e9" exitCode=0
Oct 30 22:52:16.784034 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:52:16.783666    4412 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-7476d02544/isvc-paddle-runtime-predictor-57968fb656-2z7dr" event={"ID":"7797d891-fae0-4c51-82aa-df9f03a337b1","Type":"ContainerDied","Data":"870d9bb77427ae0dbd16f600572f545c085360041003b0fcbc60aafc711ee2e9"}
Oct 30 22:52:18.583449 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:52:18.583408    4412 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-7476d02544/isvc-paddle-runtime-predictor-57968fb656-2z7dr" podUID="7797d891-fae0-4c51-82aa-df9f03a337b1" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.36:8643/healthz\": dial tcp 10.133.0.36:8643: connect: connection refused"
Oct 30 22:52:18.587915 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:52:18.587890    4412 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-7476d02544/isvc-paddle-runtime-predictor-57968fb656-2z7dr" podUID="7797d891-fae0-4c51-82aa-df9f03a337b1" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.36:8080: connect: connection refused"
Oct 30 22:52:18.701297 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:52:18.701273    4412 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-7476d02544/isvc-paddle-runtime-predictor-57968fb656-2z7dr"
Oct 30 22:52:18.715359 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:52:18.715343    4412 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/7797d891-fae0-4c51-82aa-df9f03a337b1-proxy-tls\" (UniqueName: \"kubernetes.io/secret/7797d891-fae0-4c51-82aa-df9f03a337b1-proxy-tls\") pod \"7797d891-fae0-4c51-82aa-df9f03a337b1\" (UID: \"7797d891-fae0-4c51-82aa-df9f03a337b1\") "
Oct 30 22:52:18.715430 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:52:18.715374    4412 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7797d891-fae0-4c51-82aa-df9f03a337b1-kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/7797d891-fae0-4c51-82aa-df9f03a337b1-kserve-provision-location\") pod \"7797d891-fae0-4c51-82aa-df9f03a337b1\" (UID: \"7797d891-fae0-4c51-82aa-df9f03a337b1\") "
Oct 30 22:52:18.715466 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:52:18.715450    4412 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7797d891-fae0-4c51-82aa-df9f03a337b1-kube-api-access-m24bw\" (UniqueName: \"kubernetes.io/projected/7797d891-fae0-4c51-82aa-df9f03a337b1-kube-api-access-m24bw\") pod \"7797d891-fae0-4c51-82aa-df9f03a337b1\" (UID: \"7797d891-fae0-4c51-82aa-df9f03a337b1\") "
Oct 30 22:52:18.715500 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:52:18.715489    4412 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/7797d891-fae0-4c51-82aa-df9f03a337b1-cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/7797d891-fae0-4c51-82aa-df9f03a337b1-cabundle-cert\") pod \"7797d891-fae0-4c51-82aa-df9f03a337b1\" (UID: \"7797d891-fae0-4c51-82aa-df9f03a337b1\") "
Oct 30 22:52:18.715558 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:52:18.715543    4412 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/7797d891-fae0-4c51-82aa-df9f03a337b1-kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/7797d891-fae0-4c51-82aa-df9f03a337b1-kube-rbac-proxy-sar-config\") pod \"7797d891-fae0-4c51-82aa-df9f03a337b1\" (UID: \"7797d891-fae0-4c51-82aa-df9f03a337b1\") "
Oct 30 22:52:18.715919 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:52:18.715884    4412 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7797d891-fae0-4c51-82aa-df9f03a337b1-cabundle-cert" pod "7797d891-fae0-4c51-82aa-df9f03a337b1" (UID: "7797d891-fae0-4c51-82aa-df9f03a337b1"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:52:18.716210 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:52:18.715917    4412 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7797d891-fae0-4c51-82aa-df9f03a337b1-kube-rbac-proxy-sar-config" pod "7797d891-fae0-4c51-82aa-df9f03a337b1" (UID: "7797d891-fae0-4c51-82aa-df9f03a337b1"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:52:18.717594 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:52:18.717572    4412 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7797d891-fae0-4c51-82aa-df9f03a337b1-proxy-tls" pod "7797d891-fae0-4c51-82aa-df9f03a337b1" (UID: "7797d891-fae0-4c51-82aa-df9f03a337b1"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:52:18.717711 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:52:18.717625    4412 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7797d891-fae0-4c51-82aa-df9f03a337b1-kube-api-access-m24bw" pod "7797d891-fae0-4c51-82aa-df9f03a337b1" (UID: "7797d891-fae0-4c51-82aa-df9f03a337b1"). InnerVolumeSpecName "kube-api-access-m24bw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:52:18.725088 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:52:18.725066    4412 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7797d891-fae0-4c51-82aa-df9f03a337b1-kserve-provision-location" pod "7797d891-fae0-4c51-82aa-df9f03a337b1" (UID: "7797d891-fae0-4c51-82aa-df9f03a337b1"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:52:18.791907 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:52:18.791874    4412 generic.go:356] "Generic (PLEG): container finished" podID="7797d891-fae0-4c51-82aa-df9f03a337b1" containerID="56a511707da1718542a7f6d2ba2409e68e7f32e580a1cdc3ddc8f9e3e46ef168" exitCode=0
Oct 30 22:52:18.792099 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:52:18.791908    4412 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-7476d02544/isvc-paddle-runtime-predictor-57968fb656-2z7dr" event={"ID":"7797d891-fae0-4c51-82aa-df9f03a337b1","Type":"ContainerDied","Data":"56a511707da1718542a7f6d2ba2409e68e7f32e580a1cdc3ddc8f9e3e46ef168"}
Oct 30 22:52:18.792099 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:52:18.791942    4412 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-7476d02544/isvc-paddle-runtime-predictor-57968fb656-2z7dr"
Oct 30 22:52:18.792099 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:52:18.792002    4412 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-7476d02544/isvc-paddle-runtime-predictor-57968fb656-2z7dr" event={"ID":"7797d891-fae0-4c51-82aa-df9f03a337b1","Type":"ContainerDied","Data":"eeadc94cfb58cd45f6be19edc6363e4ea793ce6b952a8c2c0688bd1d05688358"}
Oct 30 22:52:18.792099 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:52:18.792039    4412 scope.go:122] "RemoveContainer" containerID="870d9bb77427ae0dbd16f600572f545c085360041003b0fcbc60aafc711ee2e9"
Oct 30 22:52:18.800566 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:52:18.800547    4412 scope.go:122] "RemoveContainer" containerID="56a511707da1718542a7f6d2ba2409e68e7f32e580a1cdc3ddc8f9e3e46ef168"
Oct 30 22:52:18.807794 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:52:18.807775    4412 scope.go:122] "RemoveContainer" containerID="c61f15776e76125670b9560e36147c3c5331767b3f3e3b89e85c8d5c69d8d130"
Oct 30 22:52:18.814808 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:52:18.814791    4412 scope.go:122] "RemoveContainer" containerID="870d9bb77427ae0dbd16f600572f545c085360041003b0fcbc60aafc711ee2e9"
Oct 30 22:52:18.815062 ip-10-0-131-202 kubenswrapper[4412]: E1030 22:52:18.815044    4412 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"870d9bb77427ae0dbd16f600572f545c085360041003b0fcbc60aafc711ee2e9\": container with ID starting with 870d9bb77427ae0dbd16f600572f545c085360041003b0fcbc60aafc711ee2e9 not found: ID does not exist" containerID="870d9bb77427ae0dbd16f600572f545c085360041003b0fcbc60aafc711ee2e9"
Oct 30 22:52:18.815114 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:52:18.815071    4412 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"870d9bb77427ae0dbd16f600572f545c085360041003b0fcbc60aafc711ee2e9"} err="failed to get container status \"870d9bb77427ae0dbd16f600572f545c085360041003b0fcbc60aafc711ee2e9\": rpc error: code = NotFound desc = could not find container \"870d9bb77427ae0dbd16f600572f545c085360041003b0fcbc60aafc711ee2e9\": container with ID starting with 870d9bb77427ae0dbd16f600572f545c085360041003b0fcbc60aafc711ee2e9 not found: ID does not exist"
Oct 30 22:52:18.815114 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:52:18.815088    4412 scope.go:122] "RemoveContainer" containerID="56a511707da1718542a7f6d2ba2409e68e7f32e580a1cdc3ddc8f9e3e46ef168"
Oct 30 22:52:18.815348 ip-10-0-131-202 kubenswrapper[4412]: E1030 22:52:18.815332    4412 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"56a511707da1718542a7f6d2ba2409e68e7f32e580a1cdc3ddc8f9e3e46ef168\": container with ID starting with 56a511707da1718542a7f6d2ba2409e68e7f32e580a1cdc3ddc8f9e3e46ef168 not found: ID does not exist" containerID="56a511707da1718542a7f6d2ba2409e68e7f32e580a1cdc3ddc8f9e3e46ef168"
Oct 30 22:52:18.815396 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:52:18.815355    4412 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"56a511707da1718542a7f6d2ba2409e68e7f32e580a1cdc3ddc8f9e3e46ef168"} err="failed to get container status \"56a511707da1718542a7f6d2ba2409e68e7f32e580a1cdc3ddc8f9e3e46ef168\": rpc error: code = NotFound desc = could not find container \"56a511707da1718542a7f6d2ba2409e68e7f32e580a1cdc3ddc8f9e3e46ef168\": container with ID starting with 56a511707da1718542a7f6d2ba2409e68e7f32e580a1cdc3ddc8f9e3e46ef168 not found: ID does not exist"
Oct 30 22:52:18.815396 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:52:18.815370    4412 scope.go:122] "RemoveContainer" containerID="c61f15776e76125670b9560e36147c3c5331767b3f3e3b89e85c8d5c69d8d130"
Oct 30 22:52:18.815642 ip-10-0-131-202 kubenswrapper[4412]: E1030 22:52:18.815626    4412 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c61f15776e76125670b9560e36147c3c5331767b3f3e3b89e85c8d5c69d8d130\": container with ID starting with c61f15776e76125670b9560e36147c3c5331767b3f3e3b89e85c8d5c69d8d130 not found: ID does not exist" containerID="c61f15776e76125670b9560e36147c3c5331767b3f3e3b89e85c8d5c69d8d130"
Oct 30 22:52:18.815703 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:52:18.815645    4412 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c61f15776e76125670b9560e36147c3c5331767b3f3e3b89e85c8d5c69d8d130"} err="failed to get container status \"c61f15776e76125670b9560e36147c3c5331767b3f3e3b89e85c8d5c69d8d130\": rpc error: code = NotFound desc = could not find container \"c61f15776e76125670b9560e36147c3c5331767b3f3e3b89e85c8d5c69d8d130\": container with ID starting with c61f15776e76125670b9560e36147c3c5331767b3f3e3b89e85c8d5c69d8d130 not found: ID does not exist"
Oct 30 22:52:18.816946 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:52:18.816931    4412 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m24bw\" (UniqueName: \"kubernetes.io/projected/7797d891-fae0-4c51-82aa-df9f03a337b1-kube-api-access-m24bw\") on node \"ip-10-0-131-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:52:18.816990 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:52:18.816949    4412 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/7797d891-fae0-4c51-82aa-df9f03a337b1-cabundle-cert\") on node \"ip-10-0-131-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:52:18.816990 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:52:18.816959    4412 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/7797d891-fae0-4c51-82aa-df9f03a337b1-kube-rbac-proxy-sar-config\") on node \"ip-10-0-131-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:52:18.816990 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:52:18.816969    4412 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/7797d891-fae0-4c51-82aa-df9f03a337b1-proxy-tls\") on node \"ip-10-0-131-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:52:18.816990 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:52:18.816978    4412 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/7797d891-fae0-4c51-82aa-df9f03a337b1-kserve-provision-location\") on node \"ip-10-0-131-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:52:18.984570 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:52:18.984538    4412 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw1-7476d02544/isvc-paddle-runtime-predictor-57968fb656-2z7dr"]
Oct 30 22:52:18.990853 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:52:18.990828    4412 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-gw1-7476d02544/isvc-paddle-runtime-predictor-57968fb656-2z7dr"]
Oct 30 22:52:19.625869 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:52:19.625838    4412 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="7797d891-fae0-4c51-82aa-df9f03a337b1" path="/var/lib/kubelet/pods/7797d891-fae0-4c51-82aa-df9f03a337b1/volumes"
Oct 30 22:52:22.700538 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:52:22.700503    4412 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-gw1-7476d02544/isvc-paddle-v2-kserve-predictor-877ff9fb-c7btw"]
Oct 30 22:52:22.700942 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:52:22.700800    4412 cpu_manager.go:414] "containerMap: removing container" podUID="7797d891-fae0-4c51-82aa-df9f03a337b1" containerName="kserve-container"
Oct 30 22:52:22.700942 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:52:22.700816    4412 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7797d891-fae0-4c51-82aa-df9f03a337b1" containerName="kserve-container"
Oct 30 22:52:22.700942 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:52:22.700831    4412 cpu_manager.go:414] "containerMap: removing container" podUID="7797d891-fae0-4c51-82aa-df9f03a337b1" containerName="storage-initializer"
Oct 30 22:52:22.700942 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:52:22.700836    4412 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7797d891-fae0-4c51-82aa-df9f03a337b1" containerName="storage-initializer"
Oct 30 22:52:22.700942 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:52:22.700842    4412 cpu_manager.go:414] "containerMap: removing container" podUID="7797d891-fae0-4c51-82aa-df9f03a337b1" containerName="kube-rbac-proxy"
Oct 30 22:52:22.700942 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:52:22.700846    4412 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7797d891-fae0-4c51-82aa-df9f03a337b1" containerName="kube-rbac-proxy"
Oct 30 22:52:22.700942 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:52:22.700857    4412 cpu_manager.go:414] "containerMap: removing container" podUID="b5645218-816f-402f-92be-184fafc6e4de" containerName="kserve-container"
Oct 30 22:52:22.700942 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:52:22.700863    4412 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b5645218-816f-402f-92be-184fafc6e4de" containerName="kserve-container"
Oct 30 22:52:22.700942 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:52:22.700869    4412 cpu_manager.go:414] "containerMap: removing container" podUID="b5645218-816f-402f-92be-184fafc6e4de" containerName="kube-rbac-proxy"
Oct 30 22:52:22.700942 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:52:22.700873    4412 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b5645218-816f-402f-92be-184fafc6e4de" containerName="kube-rbac-proxy"
Oct 30 22:52:22.700942 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:52:22.700890    4412 cpu_manager.go:414] "containerMap: removing container" podUID="b5645218-816f-402f-92be-184fafc6e4de" containerName="storage-initializer"
Oct 30 22:52:22.700942 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:52:22.700899    4412 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b5645218-816f-402f-92be-184fafc6e4de" containerName="storage-initializer"
Oct 30 22:52:22.700942 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:52:22.700942    4412 memory_manager.go:365] "RemoveStaleState removing state" podUID="7797d891-fae0-4c51-82aa-df9f03a337b1" containerName="kube-rbac-proxy"
Oct 30 22:52:22.700942 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:52:22.700950    4412 memory_manager.go:365] "RemoveStaleState removing state" podUID="b5645218-816f-402f-92be-184fafc6e4de" containerName="kserve-container"
Oct 30 22:52:22.701378 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:52:22.700956    4412 memory_manager.go:365] "RemoveStaleState removing state" podUID="b5645218-816f-402f-92be-184fafc6e4de" containerName="kube-rbac-proxy"
Oct 30 22:52:22.701378 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:52:22.700966    4412 memory_manager.go:365] "RemoveStaleState removing state" podUID="7797d891-fae0-4c51-82aa-df9f03a337b1" containerName="kserve-container"
Oct 30 22:52:22.705371 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:52:22.705354    4412 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-7476d02544/isvc-paddle-v2-kserve-predictor-877ff9fb-c7btw"
Oct 30 22:52:22.746217 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:52:22.746187    4412 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/b2e98e5d-b30a-4277-b5c8-3086d229bea6-kserve-provision-location\") pod \"isvc-paddle-v2-kserve-predictor-877ff9fb-c7btw\" (UID: \"b2e98e5d-b30a-4277-b5c8-3086d229bea6\") " pod="e2e-gw1-7476d02544/isvc-paddle-v2-kserve-predictor-877ff9fb-c7btw"
Oct 30 22:52:22.746379 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:52:22.746225    4412 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/b2e98e5d-b30a-4277-b5c8-3086d229bea6-cabundle-cert\") pod \"isvc-paddle-v2-kserve-predictor-877ff9fb-c7btw\" (UID: \"b2e98e5d-b30a-4277-b5c8-3086d229bea6\") " pod="e2e-gw1-7476d02544/isvc-paddle-v2-kserve-predictor-877ff9fb-c7btw"
Oct 30 22:52:22.746379 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:52:22.746374    4412 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/b2e98e5d-b30a-4277-b5c8-3086d229bea6-kube-rbac-proxy-sar-config\") pod \"isvc-paddle-v2-kserve-predictor-877ff9fb-c7btw\" (UID: \"b2e98e5d-b30a-4277-b5c8-3086d229bea6\") " pod="e2e-gw1-7476d02544/isvc-paddle-v2-kserve-predictor-877ff9fb-c7btw"
Oct 30 22:52:22.746454 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:52:22.746401    4412 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5mbrr\" (UniqueName: \"kubernetes.io/projected/b2e98e5d-b30a-4277-b5c8-3086d229bea6-kube-api-access-5mbrr\") pod \"isvc-paddle-v2-kserve-predictor-877ff9fb-c7btw\" (UID: \"b2e98e5d-b30a-4277-b5c8-3086d229bea6\") " pod="e2e-gw1-7476d02544/isvc-paddle-v2-kserve-predictor-877ff9fb-c7btw"
Oct 30 22:52:22.746454 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:52:22.746431    4412 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/b2e98e5d-b30a-4277-b5c8-3086d229bea6-proxy-tls\") pod \"isvc-paddle-v2-kserve-predictor-877ff9fb-c7btw\" (UID: \"b2e98e5d-b30a-4277-b5c8-3086d229bea6\") " pod="e2e-gw1-7476d02544/isvc-paddle-v2-kserve-predictor-877ff9fb-c7btw"
Oct 30 22:52:22.759026 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:52:22.758999    4412 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw1-7476d02544\"/\"seaweedfs-s3-creds\""
Oct 30 22:52:22.808649 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:52:22.808621    4412 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw1-7476d02544\"/\"default-dockercfg-cxfkn\""
Oct 30 22:52:22.808809 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:52:22.808704    4412 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw1-7476d02544\"/\"kube-root-ca.crt\""
Oct 30 22:52:22.808809 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:52:22.808704    4412 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw1-7476d02544\"/\"odh-kserve-custom-ca-bundle\""
Oct 30 22:52:22.808809 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:52:22.808711    4412 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw1-7476d02544\"/\"openshift-service-ca.crt\""
Oct 30 22:52:22.808809 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:52:22.808734    4412 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw1-7476d02544\"/\"isvc-paddle-v2-kserve-kube-rbac-proxy-sar-config\""
Oct 30 22:52:22.808809 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:52:22.808784    4412 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw1-7476d02544\"/\"isvc-paddle-v2-kserve-predictor-serving-cert\""
Oct 30 22:52:22.810640 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:52:22.810620    4412 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw1-7476d02544/isvc-paddle-v2-kserve-predictor-877ff9fb-c7btw"]
Oct 30 22:52:22.847217 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:52:22.847193    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/b2e98e5d-b30a-4277-b5c8-3086d229bea6-kube-rbac-proxy-sar-config\") pod \"isvc-paddle-v2-kserve-predictor-877ff9fb-c7btw\" (UID: \"b2e98e5d-b30a-4277-b5c8-3086d229bea6\") " pod="e2e-gw1-7476d02544/isvc-paddle-v2-kserve-predictor-877ff9fb-c7btw"
Oct 30 22:52:22.847422 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:52:22.847222    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5mbrr\" (UniqueName: \"kubernetes.io/projected/b2e98e5d-b30a-4277-b5c8-3086d229bea6-kube-api-access-5mbrr\") pod \"isvc-paddle-v2-kserve-predictor-877ff9fb-c7btw\" (UID: \"b2e98e5d-b30a-4277-b5c8-3086d229bea6\") " pod="e2e-gw1-7476d02544/isvc-paddle-v2-kserve-predictor-877ff9fb-c7btw"
Oct 30 22:52:22.847422 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:52:22.847249    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/b2e98e5d-b30a-4277-b5c8-3086d229bea6-proxy-tls\") pod \"isvc-paddle-v2-kserve-predictor-877ff9fb-c7btw\" (UID: \"b2e98e5d-b30a-4277-b5c8-3086d229bea6\") " pod="e2e-gw1-7476d02544/isvc-paddle-v2-kserve-predictor-877ff9fb-c7btw"
Oct 30 22:52:22.847422 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:52:22.847276    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/b2e98e5d-b30a-4277-b5c8-3086d229bea6-kserve-provision-location\") pod \"isvc-paddle-v2-kserve-predictor-877ff9fb-c7btw\" (UID: \"b2e98e5d-b30a-4277-b5c8-3086d229bea6\") " pod="e2e-gw1-7476d02544/isvc-paddle-v2-kserve-predictor-877ff9fb-c7btw"
Oct 30 22:52:22.847422 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:52:22.847300    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/b2e98e5d-b30a-4277-b5c8-3086d229bea6-cabundle-cert\") pod \"isvc-paddle-v2-kserve-predictor-877ff9fb-c7btw\" (UID: \"b2e98e5d-b30a-4277-b5c8-3086d229bea6\") " pod="e2e-gw1-7476d02544/isvc-paddle-v2-kserve-predictor-877ff9fb-c7btw"
Oct 30 22:52:22.847847 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:52:22.847713    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/b2e98e5d-b30a-4277-b5c8-3086d229bea6-kserve-provision-location\") pod \"isvc-paddle-v2-kserve-predictor-877ff9fb-c7btw\" (UID: \"b2e98e5d-b30a-4277-b5c8-3086d229bea6\") " pod="e2e-gw1-7476d02544/isvc-paddle-v2-kserve-predictor-877ff9fb-c7btw"
Oct 30 22:52:22.847995 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:52:22.847974    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/b2e98e5d-b30a-4277-b5c8-3086d229bea6-cabundle-cert\") pod \"isvc-paddle-v2-kserve-predictor-877ff9fb-c7btw\" (UID: \"b2e98e5d-b30a-4277-b5c8-3086d229bea6\") " pod="e2e-gw1-7476d02544/isvc-paddle-v2-kserve-predictor-877ff9fb-c7btw"
Oct 30 22:52:22.848284 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:52:22.848258    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/b2e98e5d-b30a-4277-b5c8-3086d229bea6-kube-rbac-proxy-sar-config\") pod \"isvc-paddle-v2-kserve-predictor-877ff9fb-c7btw\" (UID: \"b2e98e5d-b30a-4277-b5c8-3086d229bea6\") " pod="e2e-gw1-7476d02544/isvc-paddle-v2-kserve-predictor-877ff9fb-c7btw"
Oct 30 22:52:22.850214 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:52:22.850196    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/b2e98e5d-b30a-4277-b5c8-3086d229bea6-proxy-tls\") pod \"isvc-paddle-v2-kserve-predictor-877ff9fb-c7btw\" (UID: \"b2e98e5d-b30a-4277-b5c8-3086d229bea6\") " pod="e2e-gw1-7476d02544/isvc-paddle-v2-kserve-predictor-877ff9fb-c7btw"
Oct 30 22:52:22.905965 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:52:22.905936    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5mbrr\" (UniqueName: \"kubernetes.io/projected/b2e98e5d-b30a-4277-b5c8-3086d229bea6-kube-api-access-5mbrr\") pod \"isvc-paddle-v2-kserve-predictor-877ff9fb-c7btw\" (UID: \"b2e98e5d-b30a-4277-b5c8-3086d229bea6\") " pod="e2e-gw1-7476d02544/isvc-paddle-v2-kserve-predictor-877ff9fb-c7btw"
Oct 30 22:52:23.022180 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:52:23.022145    4412 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-7476d02544/isvc-paddle-v2-kserve-predictor-877ff9fb-c7btw"
Oct 30 22:52:23.447921 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:52:23.447895    4412 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw1-7476d02544/isvc-paddle-v2-kserve-predictor-877ff9fb-c7btw"]
Oct 30 22:52:23.450418 ip-10-0-131-202 kubenswrapper[4412]: W1030 22:52:23.450392    4412 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb2e98e5d_b30a_4277_b5c8_3086d229bea6.slice/crio-952109d331f5462be2e4d82e8a35a11a0d88c01cd79835e4f52c103aef0df6c8 WatchSource:0}: Error finding container 952109d331f5462be2e4d82e8a35a11a0d88c01cd79835e4f52c103aef0df6c8: Status 404 returned error can't find the container with id 952109d331f5462be2e4d82e8a35a11a0d88c01cd79835e4f52c103aef0df6c8
Oct 30 22:52:23.814480 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:52:23.814437    4412 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-7476d02544/isvc-paddle-v2-kserve-predictor-877ff9fb-c7btw" event={"ID":"b2e98e5d-b30a-4277-b5c8-3086d229bea6","Type":"ContainerStarted","Data":"21ad9cf07eb792c0e8c1e3bdc3afb0a226d165a744991f5c50c88ccc6381c9f1"}
Oct 30 22:52:23.814976 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:52:23.814496    4412 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-7476d02544/isvc-paddle-v2-kserve-predictor-877ff9fb-c7btw" event={"ID":"b2e98e5d-b30a-4277-b5c8-3086d229bea6","Type":"ContainerStarted","Data":"952109d331f5462be2e4d82e8a35a11a0d88c01cd79835e4f52c103aef0df6c8"}
Oct 30 22:52:28.834387 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:52:28.834357    4412 generic.go:356] "Generic (PLEG): container finished" podID="b2e98e5d-b30a-4277-b5c8-3086d229bea6" containerID="21ad9cf07eb792c0e8c1e3bdc3afb0a226d165a744991f5c50c88ccc6381c9f1" exitCode=0
Oct 30 22:52:28.834786 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:52:28.834436    4412 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-7476d02544/isvc-paddle-v2-kserve-predictor-877ff9fb-c7btw" event={"ID":"b2e98e5d-b30a-4277-b5c8-3086d229bea6","Type":"ContainerDied","Data":"21ad9cf07eb792c0e8c1e3bdc3afb0a226d165a744991f5c50c88ccc6381c9f1"}
Oct 30 22:52:29.841579 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:52:29.841541    4412 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-7476d02544/isvc-paddle-v2-kserve-predictor-877ff9fb-c7btw" event={"ID":"b2e98e5d-b30a-4277-b5c8-3086d229bea6","Type":"ContainerStarted","Data":"3a6c6eb7d3bb22398b887269884e0771b12c0b5cab324b2cd02a000249a9d82e"}
Oct 30 22:52:29.841964 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:52:29.841621    4412 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-7476d02544/isvc-paddle-v2-kserve-predictor-877ff9fb-c7btw" event={"ID":"b2e98e5d-b30a-4277-b5c8-3086d229bea6","Type":"ContainerStarted","Data":"84bc19a91df0463b12e296483972bcfe94fcc328cb0b46c3212245eda8488304"}
Oct 30 22:52:29.841964 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:52:29.841846    4412 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw1-7476d02544/isvc-paddle-v2-kserve-predictor-877ff9fb-c7btw"
Oct 30 22:52:30.012586 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:52:30.012535    4412 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-gw1-7476d02544/isvc-paddle-v2-kserve-predictor-877ff9fb-c7btw" podStartSLOduration=3.203500847 podStartE2EDuration="8.012519654s" totalImagesPullingTime="0s" totalInitContainerRuntime="4.809018807s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:52:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:52:30.011339161 +0000 UTC m=+3570.516011114" watchObservedRunningTime="2026-10-30 22:52:30.012519654 +0000 UTC m=+3570.517191606"
Oct 30 22:52:30.845831 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:52:30.845794    4412 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw1-7476d02544/isvc-paddle-v2-kserve-predictor-877ff9fb-c7btw"
Oct 30 22:52:30.847145 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:52:30.847101    4412 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-7476d02544/isvc-paddle-v2-kserve-predictor-877ff9fb-c7btw" podUID="b2e98e5d-b30a-4277-b5c8-3086d229bea6" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.37:8080: connect: connection refused"
Oct 30 22:52:31.849657 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:52:31.849622    4412 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-7476d02544/isvc-paddle-v2-kserve-predictor-877ff9fb-c7btw" podUID="b2e98e5d-b30a-4277-b5c8-3086d229bea6" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.37:8080: connect: connection refused"
Oct 30 22:52:34.448532 ip-10-0-131-202 kubenswrapper[4412]: E1030 22:52:34.448491    4412 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bkhmz" containerName="extractor"
Oct 30 22:52:36.854230 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:52:36.854198    4412 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw1-7476d02544/isvc-paddle-v2-kserve-predictor-877ff9fb-c7btw"
Oct 30 22:52:36.854743 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:52:36.854652    4412 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-7476d02544/isvc-paddle-v2-kserve-predictor-877ff9fb-c7btw" podUID="b2e98e5d-b30a-4277-b5c8-3086d229bea6" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.37:8080: connect: connection refused"
Oct 30 22:52:46.855581 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:52:46.855533    4412 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-7476d02544/isvc-paddle-v2-kserve-predictor-877ff9fb-c7btw" podUID="b2e98e5d-b30a-4277-b5c8-3086d229bea6" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.37:8080: connect: connection refused"
Oct 30 22:52:52.852177 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:52:52.852114    4412 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-gw0-37d062d1dd/isvc-lightgbm-v2-kserve-predictor-58d5d594c4-tszn7"]
Oct 30 22:52:52.855621 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:52:52.855598    4412 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-37d062d1dd/isvc-lightgbm-v2-kserve-predictor-58d5d594c4-tszn7"
Oct 30 22:52:52.959061 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:52:52.959024    4412 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw0-37d062d1dd\"/\"seaweedfs-s3-creds\""
Oct 30 22:52:52.959279 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:52:52.959075    4412 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw0-37d062d1dd\"/\"isvc-lightgbm-v2-kserve-predictor-serving-cert\""
Oct 30 22:52:52.959279 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:52:52.959098    4412 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw0-37d062d1dd\"/\"odh-kserve-custom-ca-bundle\""
Oct 30 22:52:52.959279 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:52:52.959167    4412 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw0-37d062d1dd\"/\"openshift-service-ca.crt\""
Oct 30 22:52:52.959279 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:52:52.959171    4412 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw0-37d062d1dd\"/\"isvc-lightgbm-v2-kserve-kube-rbac-proxy-sar-config\""
Oct 30 22:52:52.959279 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:52:52.959175    4412 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw0-37d062d1dd\"/\"kube-root-ca.crt\""
Oct 30 22:52:52.959279 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:52:52.959188    4412 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw0-37d062d1dd\"/\"default-dockercfg-4v4s8\""
Oct 30 22:52:52.960657 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:52:52.960630    4412 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw0-37d062d1dd/isvc-lightgbm-v2-kserve-predictor-58d5d594c4-tszn7"]
Oct 30 22:52:53.008822 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:52:53.008786    4412 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/1b060510-0ea0-46b8-b372-2f57e6857187-kube-rbac-proxy-sar-config\") pod \"isvc-lightgbm-v2-kserve-predictor-58d5d594c4-tszn7\" (UID: \"1b060510-0ea0-46b8-b372-2f57e6857187\") " pod="e2e-gw0-37d062d1dd/isvc-lightgbm-v2-kserve-predictor-58d5d594c4-tszn7"
Oct 30 22:52:53.008822 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:52:53.008828    4412 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/1b060510-0ea0-46b8-b372-2f57e6857187-kserve-provision-location\") pod \"isvc-lightgbm-v2-kserve-predictor-58d5d594c4-tszn7\" (UID: \"1b060510-0ea0-46b8-b372-2f57e6857187\") " pod="e2e-gw0-37d062d1dd/isvc-lightgbm-v2-kserve-predictor-58d5d594c4-tszn7"
Oct 30 22:52:53.009030 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:52:53.008855    4412 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xfvfz\" (UniqueName: \"kubernetes.io/projected/1b060510-0ea0-46b8-b372-2f57e6857187-kube-api-access-xfvfz\") pod \"isvc-lightgbm-v2-kserve-predictor-58d5d594c4-tszn7\" (UID: \"1b060510-0ea0-46b8-b372-2f57e6857187\") " pod="e2e-gw0-37d062d1dd/isvc-lightgbm-v2-kserve-predictor-58d5d594c4-tszn7"
Oct 30 22:52:53.009030 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:52:53.008979    4412 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/1b060510-0ea0-46b8-b372-2f57e6857187-proxy-tls\") pod \"isvc-lightgbm-v2-kserve-predictor-58d5d594c4-tszn7\" (UID: \"1b060510-0ea0-46b8-b372-2f57e6857187\") " pod="e2e-gw0-37d062d1dd/isvc-lightgbm-v2-kserve-predictor-58d5d594c4-tszn7"
Oct 30 22:52:53.009030 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:52:53.009022    4412 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/1b060510-0ea0-46b8-b372-2f57e6857187-cabundle-cert\") pod \"isvc-lightgbm-v2-kserve-predictor-58d5d594c4-tszn7\" (UID: \"1b060510-0ea0-46b8-b372-2f57e6857187\") " pod="e2e-gw0-37d062d1dd/isvc-lightgbm-v2-kserve-predictor-58d5d594c4-tszn7"
Oct 30 22:52:53.110064 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:52:53.109978    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/1b060510-0ea0-46b8-b372-2f57e6857187-kube-rbac-proxy-sar-config\") pod \"isvc-lightgbm-v2-kserve-predictor-58d5d594c4-tszn7\" (UID: \"1b060510-0ea0-46b8-b372-2f57e6857187\") " pod="e2e-gw0-37d062d1dd/isvc-lightgbm-v2-kserve-predictor-58d5d594c4-tszn7"
Oct 30 22:52:53.110064 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:52:53.110018    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/1b060510-0ea0-46b8-b372-2f57e6857187-kserve-provision-location\") pod \"isvc-lightgbm-v2-kserve-predictor-58d5d594c4-tszn7\" (UID: \"1b060510-0ea0-46b8-b372-2f57e6857187\") " pod="e2e-gw0-37d062d1dd/isvc-lightgbm-v2-kserve-predictor-58d5d594c4-tszn7"
Oct 30 22:52:53.110064 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:52:53.110041    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xfvfz\" (UniqueName: \"kubernetes.io/projected/1b060510-0ea0-46b8-b372-2f57e6857187-kube-api-access-xfvfz\") pod \"isvc-lightgbm-v2-kserve-predictor-58d5d594c4-tszn7\" (UID: \"1b060510-0ea0-46b8-b372-2f57e6857187\") " pod="e2e-gw0-37d062d1dd/isvc-lightgbm-v2-kserve-predictor-58d5d594c4-tszn7"
Oct 30 22:52:53.110391 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:52:53.110073    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/1b060510-0ea0-46b8-b372-2f57e6857187-proxy-tls\") pod \"isvc-lightgbm-v2-kserve-predictor-58d5d594c4-tszn7\" (UID: \"1b060510-0ea0-46b8-b372-2f57e6857187\") " pod="e2e-gw0-37d062d1dd/isvc-lightgbm-v2-kserve-predictor-58d5d594c4-tszn7"
Oct 30 22:52:53.110391 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:52:53.110104    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/1b060510-0ea0-46b8-b372-2f57e6857187-cabundle-cert\") pod \"isvc-lightgbm-v2-kserve-predictor-58d5d594c4-tszn7\" (UID: \"1b060510-0ea0-46b8-b372-2f57e6857187\") " pod="e2e-gw0-37d062d1dd/isvc-lightgbm-v2-kserve-predictor-58d5d594c4-tszn7"
Oct 30 22:52:53.110500 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:52:53.110458    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/1b060510-0ea0-46b8-b372-2f57e6857187-kserve-provision-location\") pod \"isvc-lightgbm-v2-kserve-predictor-58d5d594c4-tszn7\" (UID: \"1b060510-0ea0-46b8-b372-2f57e6857187\") " pod="e2e-gw0-37d062d1dd/isvc-lightgbm-v2-kserve-predictor-58d5d594c4-tszn7"
Oct 30 22:52:53.110746 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:52:53.110724    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/1b060510-0ea0-46b8-b372-2f57e6857187-kube-rbac-proxy-sar-config\") pod \"isvc-lightgbm-v2-kserve-predictor-58d5d594c4-tszn7\" (UID: \"1b060510-0ea0-46b8-b372-2f57e6857187\") " pod="e2e-gw0-37d062d1dd/isvc-lightgbm-v2-kserve-predictor-58d5d594c4-tszn7"
Oct 30 22:52:53.110803 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:52:53.110749    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/1b060510-0ea0-46b8-b372-2f57e6857187-cabundle-cert\") pod \"isvc-lightgbm-v2-kserve-predictor-58d5d594c4-tszn7\" (UID: \"1b060510-0ea0-46b8-b372-2f57e6857187\") " pod="e2e-gw0-37d062d1dd/isvc-lightgbm-v2-kserve-predictor-58d5d594c4-tszn7"
Oct 30 22:52:53.112868 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:52:53.112838    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/1b060510-0ea0-46b8-b372-2f57e6857187-proxy-tls\") pod \"isvc-lightgbm-v2-kserve-predictor-58d5d594c4-tszn7\" (UID: \"1b060510-0ea0-46b8-b372-2f57e6857187\") " pod="e2e-gw0-37d062d1dd/isvc-lightgbm-v2-kserve-predictor-58d5d594c4-tszn7"
Oct 30 22:52:53.169006 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:52:53.168971    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xfvfz\" (UniqueName: \"kubernetes.io/projected/1b060510-0ea0-46b8-b372-2f57e6857187-kube-api-access-xfvfz\") pod \"isvc-lightgbm-v2-kserve-predictor-58d5d594c4-tszn7\" (UID: \"1b060510-0ea0-46b8-b372-2f57e6857187\") " pod="e2e-gw0-37d062d1dd/isvc-lightgbm-v2-kserve-predictor-58d5d594c4-tszn7"
Oct 30 22:52:53.172653 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:52:53.172623    4412 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-37d062d1dd/isvc-lightgbm-v2-kserve-predictor-58d5d594c4-tszn7"
Oct 30 22:52:53.594379 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:52:53.594273    4412 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw0-37d062d1dd/isvc-lightgbm-v2-kserve-predictor-58d5d594c4-tszn7"]
Oct 30 22:52:53.597109 ip-10-0-131-202 kubenswrapper[4412]: W1030 22:52:53.597076    4412 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1b060510_0ea0_46b8_b372_2f57e6857187.slice/crio-5b725d0dafba5f172d256f434783596598029ca41fd570d580d321a2969256fe WatchSource:0}: Error finding container 5b725d0dafba5f172d256f434783596598029ca41fd570d580d321a2969256fe: Status 404 returned error can't find the container with id 5b725d0dafba5f172d256f434783596598029ca41fd570d580d321a2969256fe
Oct 30 22:52:53.933999 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:52:53.933913    4412 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-37d062d1dd/isvc-lightgbm-v2-kserve-predictor-58d5d594c4-tszn7" event={"ID":"1b060510-0ea0-46b8-b372-2f57e6857187","Type":"ContainerStarted","Data":"49e433ac4ed3995e005fd25e97f420ac95f092c50152045807d99faccb2ec4bf"}
Oct 30 22:52:53.934399 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:52:53.934001    4412 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-37d062d1dd/isvc-lightgbm-v2-kserve-predictor-58d5d594c4-tszn7" event={"ID":"1b060510-0ea0-46b8-b372-2f57e6857187","Type":"ContainerStarted","Data":"5b725d0dafba5f172d256f434783596598029ca41fd570d580d321a2969256fe"}
Oct 30 22:52:56.855192 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:52:56.855155    4412 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-7476d02544/isvc-paddle-v2-kserve-predictor-877ff9fb-c7btw" podUID="b2e98e5d-b30a-4277-b5c8-3086d229bea6" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.37:8080: connect: connection refused"
Oct 30 22:52:57.951133 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:52:57.951039    4412 generic.go:356] "Generic (PLEG): container finished" podID="1b060510-0ea0-46b8-b372-2f57e6857187" containerID="49e433ac4ed3995e005fd25e97f420ac95f092c50152045807d99faccb2ec4bf" exitCode=0
Oct 30 22:52:57.951133 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:52:57.951076    4412 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-37d062d1dd/isvc-lightgbm-v2-kserve-predictor-58d5d594c4-tszn7" event={"ID":"1b060510-0ea0-46b8-b372-2f57e6857187","Type":"ContainerDied","Data":"49e433ac4ed3995e005fd25e97f420ac95f092c50152045807d99faccb2ec4bf"}
Oct 30 22:52:58.958012 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:52:58.957982    4412 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-37d062d1dd/isvc-lightgbm-v2-kserve-predictor-58d5d594c4-tszn7" event={"ID":"1b060510-0ea0-46b8-b372-2f57e6857187","Type":"ContainerStarted","Data":"c68316cfd33e5b7beecc4d80a034b2c5dced4deb1268780543f1cfdc3dfa81b1"}
Oct 30 22:52:58.958534 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:52:58.958050    4412 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-37d062d1dd/isvc-lightgbm-v2-kserve-predictor-58d5d594c4-tszn7" event={"ID":"1b060510-0ea0-46b8-b372-2f57e6857187","Type":"ContainerStarted","Data":"a9480b62838e13b98b9922e21163046b438cb73e34db0c11b6a279950316147b"}
Oct 30 22:52:58.958534 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:52:58.958349    4412 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw0-37d062d1dd/isvc-lightgbm-v2-kserve-predictor-58d5d594c4-tszn7"
Oct 30 22:52:58.958534 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:52:58.958397    4412 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw0-37d062d1dd/isvc-lightgbm-v2-kserve-predictor-58d5d594c4-tszn7"
Oct 30 22:52:58.959937 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:52:58.959908    4412 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-37d062d1dd/isvc-lightgbm-v2-kserve-predictor-58d5d594c4-tszn7" podUID="1b060510-0ea0-46b8-b372-2f57e6857187" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.38:8080: connect: connection refused"
Oct 30 22:52:59.129709 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:52:59.129655    4412 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-gw0-37d062d1dd/isvc-lightgbm-v2-kserve-predictor-58d5d594c4-tszn7" podStartSLOduration=3.300168347 podStartE2EDuration="7.129639014s" totalImagesPullingTime="0s" totalInitContainerRuntime="3.829470667s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:52:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:52:59.128710957 +0000 UTC m=+3599.633382911" watchObservedRunningTime="2026-10-30 22:52:59.129639014 +0000 UTC m=+3599.634310966"
Oct 30 22:52:59.962294 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:52:59.962251    4412 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-37d062d1dd/isvc-lightgbm-v2-kserve-predictor-58d5d594c4-tszn7" podUID="1b060510-0ea0-46b8-b372-2f57e6857187" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.38:8080: connect: connection refused"
Oct 30 22:53:04.449492 ip-10-0-131-202 kubenswrapper[4412]: E1030 22:53:04.449452    4412 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bkhmz" containerName="extractor"
Oct 30 22:53:04.967413 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:53:04.967387    4412 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw0-37d062d1dd/isvc-lightgbm-v2-kserve-predictor-58d5d594c4-tszn7"
Oct 30 22:53:04.968039 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:53:04.968004    4412 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-37d062d1dd/isvc-lightgbm-v2-kserve-predictor-58d5d594c4-tszn7" podUID="1b060510-0ea0-46b8-b372-2f57e6857187" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.38:8080: connect: connection refused"
Oct 30 22:53:06.854755 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:53:06.854714    4412 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-7476d02544/isvc-paddle-v2-kserve-predictor-877ff9fb-c7btw" podUID="b2e98e5d-b30a-4277-b5c8-3086d229bea6" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.37:8080: connect: connection refused"
Oct 30 22:53:14.968849 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:53:14.968822    4412 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw0-37d062d1dd/isvc-lightgbm-v2-kserve-predictor-58d5d594c4-tszn7"
Oct 30 22:53:16.855296 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:53:16.855267    4412 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw1-7476d02544/isvc-paddle-v2-kserve-predictor-877ff9fb-c7btw"
Oct 30 22:53:23.037553 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:53:23.037520    4412 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw0-37d062d1dd/isvc-lightgbm-v2-kserve-predictor-58d5d594c4-tszn7"]
Oct 30 22:53:23.037980 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:53:23.037852    4412 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw0-37d062d1dd/isvc-lightgbm-v2-kserve-predictor-58d5d594c4-tszn7" podUID="1b060510-0ea0-46b8-b372-2f57e6857187" containerName="kserve-container" containerID="cri-o://a9480b62838e13b98b9922e21163046b438cb73e34db0c11b6a279950316147b" gracePeriod=30
Oct 30 22:53:23.037980 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:53:23.037914    4412 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw0-37d062d1dd/isvc-lightgbm-v2-kserve-predictor-58d5d594c4-tszn7" podUID="1b060510-0ea0-46b8-b372-2f57e6857187" containerName="kube-rbac-proxy" containerID="cri-o://c68316cfd33e5b7beecc4d80a034b2c5dced4deb1268780543f1cfdc3dfa81b1" gracePeriod=30
Oct 30 22:53:23.780603 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:53:23.780580    4412 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-37d062d1dd/isvc-lightgbm-v2-kserve-predictor-58d5d594c4-tszn7"
Oct 30 22:53:23.869526 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:53:23.869496    4412 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1b060510-0ea0-46b8-b372-2f57e6857187-kube-api-access-xfvfz\" (UniqueName: \"kubernetes.io/projected/1b060510-0ea0-46b8-b372-2f57e6857187-kube-api-access-xfvfz\") pod \"1b060510-0ea0-46b8-b372-2f57e6857187\" (UID: \"1b060510-0ea0-46b8-b372-2f57e6857187\") "
Oct 30 22:53:23.869719 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:53:23.869541    4412 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1b060510-0ea0-46b8-b372-2f57e6857187-kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/1b060510-0ea0-46b8-b372-2f57e6857187-kube-rbac-proxy-sar-config\") pod \"1b060510-0ea0-46b8-b372-2f57e6857187\" (UID: \"1b060510-0ea0-46b8-b372-2f57e6857187\") "
Oct 30 22:53:23.869719 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:53:23.869560    4412 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1b060510-0ea0-46b8-b372-2f57e6857187-cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/1b060510-0ea0-46b8-b372-2f57e6857187-cabundle-cert\") pod \"1b060510-0ea0-46b8-b372-2f57e6857187\" (UID: \"1b060510-0ea0-46b8-b372-2f57e6857187\") "
Oct 30 22:53:23.869719 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:53:23.869586    4412 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/1b060510-0ea0-46b8-b372-2f57e6857187-proxy-tls\" (UniqueName: \"kubernetes.io/secret/1b060510-0ea0-46b8-b372-2f57e6857187-proxy-tls\") pod \"1b060510-0ea0-46b8-b372-2f57e6857187\" (UID: \"1b060510-0ea0-46b8-b372-2f57e6857187\") "
Oct 30 22:53:23.869719 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:53:23.869639    4412 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1b060510-0ea0-46b8-b372-2f57e6857187-kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/1b060510-0ea0-46b8-b372-2f57e6857187-kserve-provision-location\") pod \"1b060510-0ea0-46b8-b372-2f57e6857187\" (UID: \"1b060510-0ea0-46b8-b372-2f57e6857187\") "
Oct 30 22:53:23.870003 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:53:23.869976    4412 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1b060510-0ea0-46b8-b372-2f57e6857187-kube-rbac-proxy-sar-config" pod "1b060510-0ea0-46b8-b372-2f57e6857187" (UID: "1b060510-0ea0-46b8-b372-2f57e6857187"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:53:23.870003 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:53:23.869993    4412 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1b060510-0ea0-46b8-b372-2f57e6857187-cabundle-cert" pod "1b060510-0ea0-46b8-b372-2f57e6857187" (UID: "1b060510-0ea0-46b8-b372-2f57e6857187"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:53:23.870088 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:53:23.869991    4412 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1b060510-0ea0-46b8-b372-2f57e6857187-kserve-provision-location" pod "1b060510-0ea0-46b8-b372-2f57e6857187" (UID: "1b060510-0ea0-46b8-b372-2f57e6857187"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:53:23.871898 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:53:23.871876    4412 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1b060510-0ea0-46b8-b372-2f57e6857187-proxy-tls" pod "1b060510-0ea0-46b8-b372-2f57e6857187" (UID: "1b060510-0ea0-46b8-b372-2f57e6857187"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:53:23.872006 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:53:23.871918    4412 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1b060510-0ea0-46b8-b372-2f57e6857187-kube-api-access-xfvfz" pod "1b060510-0ea0-46b8-b372-2f57e6857187" (UID: "1b060510-0ea0-46b8-b372-2f57e6857187"). InnerVolumeSpecName "kube-api-access-xfvfz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:53:23.970818 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:53:23.970789    4412 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/1b060510-0ea0-46b8-b372-2f57e6857187-kserve-provision-location\") on node \"ip-10-0-131-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:53:23.970818 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:53:23.970817    4412 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xfvfz\" (UniqueName: \"kubernetes.io/projected/1b060510-0ea0-46b8-b372-2f57e6857187-kube-api-access-xfvfz\") on node \"ip-10-0-131-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:53:23.971020 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:53:23.970827    4412 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/1b060510-0ea0-46b8-b372-2f57e6857187-kube-rbac-proxy-sar-config\") on node \"ip-10-0-131-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:53:23.971020 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:53:23.970837    4412 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/1b060510-0ea0-46b8-b372-2f57e6857187-cabundle-cert\") on node \"ip-10-0-131-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:53:23.971020 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:53:23.970848    4412 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/1b060510-0ea0-46b8-b372-2f57e6857187-proxy-tls\") on node \"ip-10-0-131-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:53:24.054835 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:53:24.054804    4412 generic.go:356] "Generic (PLEG): container finished" podID="1b060510-0ea0-46b8-b372-2f57e6857187" containerID="c68316cfd33e5b7beecc4d80a034b2c5dced4deb1268780543f1cfdc3dfa81b1" exitCode=0
Oct 30 22:53:24.054835 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:53:24.054829    4412 generic.go:356] "Generic (PLEG): container finished" podID="1b060510-0ea0-46b8-b372-2f57e6857187" containerID="a9480b62838e13b98b9922e21163046b438cb73e34db0c11b6a279950316147b" exitCode=0
Oct 30 22:53:24.055292 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:53:24.054876    4412 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-37d062d1dd/isvc-lightgbm-v2-kserve-predictor-58d5d594c4-tszn7" event={"ID":"1b060510-0ea0-46b8-b372-2f57e6857187","Type":"ContainerDied","Data":"c68316cfd33e5b7beecc4d80a034b2c5dced4deb1268780543f1cfdc3dfa81b1"}
Oct 30 22:53:24.055292 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:53:24.054881    4412 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-37d062d1dd/isvc-lightgbm-v2-kserve-predictor-58d5d594c4-tszn7"
Oct 30 22:53:24.055292 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:53:24.054932    4412 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-37d062d1dd/isvc-lightgbm-v2-kserve-predictor-58d5d594c4-tszn7" event={"ID":"1b060510-0ea0-46b8-b372-2f57e6857187","Type":"ContainerDied","Data":"a9480b62838e13b98b9922e21163046b438cb73e34db0c11b6a279950316147b"}
Oct 30 22:53:24.055292 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:53:24.054955    4412 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-37d062d1dd/isvc-lightgbm-v2-kserve-predictor-58d5d594c4-tszn7" event={"ID":"1b060510-0ea0-46b8-b372-2f57e6857187","Type":"ContainerDied","Data":"5b725d0dafba5f172d256f434783596598029ca41fd570d580d321a2969256fe"}
Oct 30 22:53:24.055292 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:53:24.054978    4412 scope.go:122] "RemoveContainer" containerID="c68316cfd33e5b7beecc4d80a034b2c5dced4deb1268780543f1cfdc3dfa81b1"
Oct 30 22:53:24.064257 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:53:24.064240    4412 scope.go:122] "RemoveContainer" containerID="a9480b62838e13b98b9922e21163046b438cb73e34db0c11b6a279950316147b"
Oct 30 22:53:24.071558 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:53:24.071536    4412 scope.go:122] "RemoveContainer" containerID="49e433ac4ed3995e005fd25e97f420ac95f092c50152045807d99faccb2ec4bf"
Oct 30 22:53:24.078772 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:53:24.078753    4412 scope.go:122] "RemoveContainer" containerID="c68316cfd33e5b7beecc4d80a034b2c5dced4deb1268780543f1cfdc3dfa81b1"
Oct 30 22:53:24.079034 ip-10-0-131-202 kubenswrapper[4412]: E1030 22:53:24.079014    4412 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c68316cfd33e5b7beecc4d80a034b2c5dced4deb1268780543f1cfdc3dfa81b1\": container with ID starting with c68316cfd33e5b7beecc4d80a034b2c5dced4deb1268780543f1cfdc3dfa81b1 not found: ID does not exist" containerID="c68316cfd33e5b7beecc4d80a034b2c5dced4deb1268780543f1cfdc3dfa81b1"
Oct 30 22:53:24.079081 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:53:24.079044    4412 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c68316cfd33e5b7beecc4d80a034b2c5dced4deb1268780543f1cfdc3dfa81b1"} err="failed to get container status \"c68316cfd33e5b7beecc4d80a034b2c5dced4deb1268780543f1cfdc3dfa81b1\": rpc error: code = NotFound desc = could not find container \"c68316cfd33e5b7beecc4d80a034b2c5dced4deb1268780543f1cfdc3dfa81b1\": container with ID starting with c68316cfd33e5b7beecc4d80a034b2c5dced4deb1268780543f1cfdc3dfa81b1 not found: ID does not exist"
Oct 30 22:53:24.079081 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:53:24.079064    4412 scope.go:122] "RemoveContainer" containerID="a9480b62838e13b98b9922e21163046b438cb73e34db0c11b6a279950316147b"
Oct 30 22:53:24.079334 ip-10-0-131-202 kubenswrapper[4412]: E1030 22:53:24.079315    4412 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a9480b62838e13b98b9922e21163046b438cb73e34db0c11b6a279950316147b\": container with ID starting with a9480b62838e13b98b9922e21163046b438cb73e34db0c11b6a279950316147b not found: ID does not exist" containerID="a9480b62838e13b98b9922e21163046b438cb73e34db0c11b6a279950316147b"
Oct 30 22:53:24.079382 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:53:24.079341    4412 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a9480b62838e13b98b9922e21163046b438cb73e34db0c11b6a279950316147b"} err="failed to get container status \"a9480b62838e13b98b9922e21163046b438cb73e34db0c11b6a279950316147b\": rpc error: code = NotFound desc = could not find container \"a9480b62838e13b98b9922e21163046b438cb73e34db0c11b6a279950316147b\": container with ID starting with a9480b62838e13b98b9922e21163046b438cb73e34db0c11b6a279950316147b not found: ID does not exist"
Oct 30 22:53:24.079382 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:53:24.079357    4412 scope.go:122] "RemoveContainer" containerID="49e433ac4ed3995e005fd25e97f420ac95f092c50152045807d99faccb2ec4bf"
Oct 30 22:53:24.079602 ip-10-0-131-202 kubenswrapper[4412]: E1030 22:53:24.079581    4412 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"49e433ac4ed3995e005fd25e97f420ac95f092c50152045807d99faccb2ec4bf\": container with ID starting with 49e433ac4ed3995e005fd25e97f420ac95f092c50152045807d99faccb2ec4bf not found: ID does not exist" containerID="49e433ac4ed3995e005fd25e97f420ac95f092c50152045807d99faccb2ec4bf"
Oct 30 22:53:24.079648 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:53:24.079608    4412 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"49e433ac4ed3995e005fd25e97f420ac95f092c50152045807d99faccb2ec4bf"} err="failed to get container status \"49e433ac4ed3995e005fd25e97f420ac95f092c50152045807d99faccb2ec4bf\": rpc error: code = NotFound desc = could not find container \"49e433ac4ed3995e005fd25e97f420ac95f092c50152045807d99faccb2ec4bf\": container with ID starting with 49e433ac4ed3995e005fd25e97f420ac95f092c50152045807d99faccb2ec4bf not found: ID does not exist"
Oct 30 22:53:24.079648 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:53:24.079622    4412 scope.go:122] "RemoveContainer" containerID="c68316cfd33e5b7beecc4d80a034b2c5dced4deb1268780543f1cfdc3dfa81b1"
Oct 30 22:53:24.079839 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:53:24.079823    4412 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c68316cfd33e5b7beecc4d80a034b2c5dced4deb1268780543f1cfdc3dfa81b1"} err="failed to get container status \"c68316cfd33e5b7beecc4d80a034b2c5dced4deb1268780543f1cfdc3dfa81b1\": rpc error: code = NotFound desc = could not find container \"c68316cfd33e5b7beecc4d80a034b2c5dced4deb1268780543f1cfdc3dfa81b1\": container with ID starting with c68316cfd33e5b7beecc4d80a034b2c5dced4deb1268780543f1cfdc3dfa81b1 not found: ID does not exist"
Oct 30 22:53:24.079884 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:53:24.079839    4412 scope.go:122] "RemoveContainer" containerID="a9480b62838e13b98b9922e21163046b438cb73e34db0c11b6a279950316147b"
Oct 30 22:53:24.080081 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:53:24.080061    4412 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a9480b62838e13b98b9922e21163046b438cb73e34db0c11b6a279950316147b"} err="failed to get container status \"a9480b62838e13b98b9922e21163046b438cb73e34db0c11b6a279950316147b\": rpc error: code = NotFound desc = could not find container \"a9480b62838e13b98b9922e21163046b438cb73e34db0c11b6a279950316147b\": container with ID starting with a9480b62838e13b98b9922e21163046b438cb73e34db0c11b6a279950316147b not found: ID does not exist"
Oct 30 22:53:24.080171 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:53:24.080082    4412 scope.go:122] "RemoveContainer" containerID="49e433ac4ed3995e005fd25e97f420ac95f092c50152045807d99faccb2ec4bf"
Oct 30 22:53:24.080347 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:53:24.080324    4412 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"49e433ac4ed3995e005fd25e97f420ac95f092c50152045807d99faccb2ec4bf"} err="failed to get container status \"49e433ac4ed3995e005fd25e97f420ac95f092c50152045807d99faccb2ec4bf\": rpc error: code = NotFound desc = could not find container \"49e433ac4ed3995e005fd25e97f420ac95f092c50152045807d99faccb2ec4bf\": container with ID starting with 49e433ac4ed3995e005fd25e97f420ac95f092c50152045807d99faccb2ec4bf not found: ID does not exist"
Oct 30 22:53:24.222286 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:53:24.222210    4412 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw0-37d062d1dd/isvc-lightgbm-v2-kserve-predictor-58d5d594c4-tszn7"]
Oct 30 22:53:24.225641 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:53:24.225618    4412 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-gw0-37d062d1dd/isvc-lightgbm-v2-kserve-predictor-58d5d594c4-tszn7"]
Oct 30 22:53:24.512869 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:53:24.512778    4412 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw1-7476d02544/isvc-paddle-v2-kserve-predictor-877ff9fb-c7btw"]
Oct 30 22:53:24.513078 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:53:24.513049    4412 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw1-7476d02544/isvc-paddle-v2-kserve-predictor-877ff9fb-c7btw" podUID="b2e98e5d-b30a-4277-b5c8-3086d229bea6" containerName="kserve-container" containerID="cri-o://84bc19a91df0463b12e296483972bcfe94fcc328cb0b46c3212245eda8488304" gracePeriod=30
Oct 30 22:53:24.513186 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:53:24.513114    4412 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw1-7476d02544/isvc-paddle-v2-kserve-predictor-877ff9fb-c7btw" podUID="b2e98e5d-b30a-4277-b5c8-3086d229bea6" containerName="kube-rbac-proxy" containerID="cri-o://3a6c6eb7d3bb22398b887269884e0771b12c0b5cab324b2cd02a000249a9d82e" gracePeriod=30
Oct 30 22:53:25.061674 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:53:25.061642    4412 generic.go:356] "Generic (PLEG): container finished" podID="b2e98e5d-b30a-4277-b5c8-3086d229bea6" containerID="3a6c6eb7d3bb22398b887269884e0771b12c0b5cab324b2cd02a000249a9d82e" exitCode=0
Oct 30 22:53:25.062072 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:53:25.061698    4412 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-7476d02544/isvc-paddle-v2-kserve-predictor-877ff9fb-c7btw" event={"ID":"b2e98e5d-b30a-4277-b5c8-3086d229bea6","Type":"ContainerDied","Data":"3a6c6eb7d3bb22398b887269884e0771b12c0b5cab324b2cd02a000249a9d82e"}
Oct 30 22:53:25.625401 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:53:25.625371    4412 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1b060510-0ea0-46b8-b372-2f57e6857187" path="/var/lib/kubelet/pods/1b060510-0ea0-46b8-b372-2f57e6857187/volumes"
Oct 30 22:53:26.850497 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:53:26.850428    4412 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-7476d02544/isvc-paddle-v2-kserve-predictor-877ff9fb-c7btw" podUID="b2e98e5d-b30a-4277-b5c8-3086d229bea6" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.37:8643/healthz\": dial tcp 10.133.0.37:8643: connect: connection refused"
Oct 30 22:53:26.854757 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:53:26.854726    4412 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-7476d02544/isvc-paddle-v2-kserve-predictor-877ff9fb-c7btw" podUID="b2e98e5d-b30a-4277-b5c8-3086d229bea6" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.37:8080: connect: connection refused"
Oct 30 22:53:27.556630 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:53:27.556603    4412 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-7476d02544/isvc-paddle-v2-kserve-predictor-877ff9fb-c7btw"
Oct 30 22:53:27.602087 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:53:27.602011    4412 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b2e98e5d-b30a-4277-b5c8-3086d229bea6-kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/b2e98e5d-b30a-4277-b5c8-3086d229bea6-kserve-provision-location\") pod \"b2e98e5d-b30a-4277-b5c8-3086d229bea6\" (UID: \"b2e98e5d-b30a-4277-b5c8-3086d229bea6\") "
Oct 30 22:53:27.602087 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:53:27.602057    4412 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b2e98e5d-b30a-4277-b5c8-3086d229bea6-kube-api-access-5mbrr\" (UniqueName: \"kubernetes.io/projected/b2e98e5d-b30a-4277-b5c8-3086d229bea6-kube-api-access-5mbrr\") pod \"b2e98e5d-b30a-4277-b5c8-3086d229bea6\" (UID: \"b2e98e5d-b30a-4277-b5c8-3086d229bea6\") "
Oct 30 22:53:27.602293 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:53:27.602110    4412 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/b2e98e5d-b30a-4277-b5c8-3086d229bea6-proxy-tls\" (UniqueName: \"kubernetes.io/secret/b2e98e5d-b30a-4277-b5c8-3086d229bea6-proxy-tls\") pod \"b2e98e5d-b30a-4277-b5c8-3086d229bea6\" (UID: \"b2e98e5d-b30a-4277-b5c8-3086d229bea6\") "
Oct 30 22:53:27.602293 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:53:27.602188    4412 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b2e98e5d-b30a-4277-b5c8-3086d229bea6-cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/b2e98e5d-b30a-4277-b5c8-3086d229bea6-cabundle-cert\") pod \"b2e98e5d-b30a-4277-b5c8-3086d229bea6\" (UID: \"b2e98e5d-b30a-4277-b5c8-3086d229bea6\") "
Oct 30 22:53:27.602293 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:53:27.602217    4412 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b2e98e5d-b30a-4277-b5c8-3086d229bea6-kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/b2e98e5d-b30a-4277-b5c8-3086d229bea6-kube-rbac-proxy-sar-config\") pod \"b2e98e5d-b30a-4277-b5c8-3086d229bea6\" (UID: \"b2e98e5d-b30a-4277-b5c8-3086d229bea6\") "
Oct 30 22:53:27.602601 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:53:27.602571    4412 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b2e98e5d-b30a-4277-b5c8-3086d229bea6-cabundle-cert" pod "b2e98e5d-b30a-4277-b5c8-3086d229bea6" (UID: "b2e98e5d-b30a-4277-b5c8-3086d229bea6"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:53:27.602713 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:53:27.602599    4412 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b2e98e5d-b30a-4277-b5c8-3086d229bea6-kube-rbac-proxy-sar-config" pod "b2e98e5d-b30a-4277-b5c8-3086d229bea6" (UID: "b2e98e5d-b30a-4277-b5c8-3086d229bea6"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:53:27.604458 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:53:27.604438    4412 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b2e98e5d-b30a-4277-b5c8-3086d229bea6-proxy-tls" pod "b2e98e5d-b30a-4277-b5c8-3086d229bea6" (UID: "b2e98e5d-b30a-4277-b5c8-3086d229bea6"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:53:27.604692 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:53:27.604677    4412 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b2e98e5d-b30a-4277-b5c8-3086d229bea6-kube-api-access-5mbrr" pod "b2e98e5d-b30a-4277-b5c8-3086d229bea6" (UID: "b2e98e5d-b30a-4277-b5c8-3086d229bea6"). InnerVolumeSpecName "kube-api-access-5mbrr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:53:27.612760 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:53:27.612735    4412 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b2e98e5d-b30a-4277-b5c8-3086d229bea6-kserve-provision-location" pod "b2e98e5d-b30a-4277-b5c8-3086d229bea6" (UID: "b2e98e5d-b30a-4277-b5c8-3086d229bea6"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:53:27.703558 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:53:27.703525    4412 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/b2e98e5d-b30a-4277-b5c8-3086d229bea6-kserve-provision-location\") on node \"ip-10-0-131-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:53:27.703558 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:53:27.703553    4412 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5mbrr\" (UniqueName: \"kubernetes.io/projected/b2e98e5d-b30a-4277-b5c8-3086d229bea6-kube-api-access-5mbrr\") on node \"ip-10-0-131-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:53:27.703558 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:53:27.703565    4412 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/b2e98e5d-b30a-4277-b5c8-3086d229bea6-proxy-tls\") on node \"ip-10-0-131-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:53:27.703794 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:53:27.703573    4412 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/b2e98e5d-b30a-4277-b5c8-3086d229bea6-cabundle-cert\") on node \"ip-10-0-131-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:53:27.703794 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:53:27.703583    4412 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/b2e98e5d-b30a-4277-b5c8-3086d229bea6-kube-rbac-proxy-sar-config\") on node \"ip-10-0-131-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:53:28.075025 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:53:28.074994    4412 generic.go:356] "Generic (PLEG): container finished" podID="b2e98e5d-b30a-4277-b5c8-3086d229bea6" containerID="84bc19a91df0463b12e296483972bcfe94fcc328cb0b46c3212245eda8488304" exitCode=0
Oct 30 22:53:28.075520 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:53:28.075029    4412 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-7476d02544/isvc-paddle-v2-kserve-predictor-877ff9fb-c7btw" event={"ID":"b2e98e5d-b30a-4277-b5c8-3086d229bea6","Type":"ContainerDied","Data":"84bc19a91df0463b12e296483972bcfe94fcc328cb0b46c3212245eda8488304"}
Oct 30 22:53:28.075520 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:53:28.075057    4412 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-7476d02544/isvc-paddle-v2-kserve-predictor-877ff9fb-c7btw"
Oct 30 22:53:28.075520 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:53:28.075080    4412 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-7476d02544/isvc-paddle-v2-kserve-predictor-877ff9fb-c7btw" event={"ID":"b2e98e5d-b30a-4277-b5c8-3086d229bea6","Type":"ContainerDied","Data":"952109d331f5462be2e4d82e8a35a11a0d88c01cd79835e4f52c103aef0df6c8"}
Oct 30 22:53:28.075520 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:53:28.075105    4412 scope.go:122] "RemoveContainer" containerID="3a6c6eb7d3bb22398b887269884e0771b12c0b5cab324b2cd02a000249a9d82e"
Oct 30 22:53:28.083105 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:53:28.083081    4412 scope.go:122] "RemoveContainer" containerID="84bc19a91df0463b12e296483972bcfe94fcc328cb0b46c3212245eda8488304"
Oct 30 22:53:28.090472 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:53:28.090455    4412 scope.go:122] "RemoveContainer" containerID="21ad9cf07eb792c0e8c1e3bdc3afb0a226d165a744991f5c50c88ccc6381c9f1"
Oct 30 22:53:28.097643 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:53:28.097608    4412 scope.go:122] "RemoveContainer" containerID="3a6c6eb7d3bb22398b887269884e0771b12c0b5cab324b2cd02a000249a9d82e"
Oct 30 22:53:28.097874 ip-10-0-131-202 kubenswrapper[4412]: E1030 22:53:28.097851    4412 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3a6c6eb7d3bb22398b887269884e0771b12c0b5cab324b2cd02a000249a9d82e\": container with ID starting with 3a6c6eb7d3bb22398b887269884e0771b12c0b5cab324b2cd02a000249a9d82e not found: ID does not exist" containerID="3a6c6eb7d3bb22398b887269884e0771b12c0b5cab324b2cd02a000249a9d82e"
Oct 30 22:53:28.097959 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:53:28.097882    4412 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3a6c6eb7d3bb22398b887269884e0771b12c0b5cab324b2cd02a000249a9d82e"} err="failed to get container status \"3a6c6eb7d3bb22398b887269884e0771b12c0b5cab324b2cd02a000249a9d82e\": rpc error: code = NotFound desc = could not find container \"3a6c6eb7d3bb22398b887269884e0771b12c0b5cab324b2cd02a000249a9d82e\": container with ID starting with 3a6c6eb7d3bb22398b887269884e0771b12c0b5cab324b2cd02a000249a9d82e not found: ID does not exist"
Oct 30 22:53:28.097959 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:53:28.097902    4412 scope.go:122] "RemoveContainer" containerID="84bc19a91df0463b12e296483972bcfe94fcc328cb0b46c3212245eda8488304"
Oct 30 22:53:28.098110 ip-10-0-131-202 kubenswrapper[4412]: E1030 22:53:28.098093    4412 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"84bc19a91df0463b12e296483972bcfe94fcc328cb0b46c3212245eda8488304\": container with ID starting with 84bc19a91df0463b12e296483972bcfe94fcc328cb0b46c3212245eda8488304 not found: ID does not exist" containerID="84bc19a91df0463b12e296483972bcfe94fcc328cb0b46c3212245eda8488304"
Oct 30 22:53:28.098168 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:53:28.098130    4412 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"84bc19a91df0463b12e296483972bcfe94fcc328cb0b46c3212245eda8488304"} err="failed to get container status \"84bc19a91df0463b12e296483972bcfe94fcc328cb0b46c3212245eda8488304\": rpc error: code = NotFound desc = could not find container \"84bc19a91df0463b12e296483972bcfe94fcc328cb0b46c3212245eda8488304\": container with ID starting with 84bc19a91df0463b12e296483972bcfe94fcc328cb0b46c3212245eda8488304 not found: ID does not exist"
Oct 30 22:53:28.098168 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:53:28.098147    4412 scope.go:122] "RemoveContainer" containerID="21ad9cf07eb792c0e8c1e3bdc3afb0a226d165a744991f5c50c88ccc6381c9f1"
Oct 30 22:53:28.098340 ip-10-0-131-202 kubenswrapper[4412]: E1030 22:53:28.098324    4412 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"21ad9cf07eb792c0e8c1e3bdc3afb0a226d165a744991f5c50c88ccc6381c9f1\": container with ID starting with 21ad9cf07eb792c0e8c1e3bdc3afb0a226d165a744991f5c50c88ccc6381c9f1 not found: ID does not exist" containerID="21ad9cf07eb792c0e8c1e3bdc3afb0a226d165a744991f5c50c88ccc6381c9f1"
Oct 30 22:53:28.098379 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:53:28.098345    4412 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"21ad9cf07eb792c0e8c1e3bdc3afb0a226d165a744991f5c50c88ccc6381c9f1"} err="failed to get container status \"21ad9cf07eb792c0e8c1e3bdc3afb0a226d165a744991f5c50c88ccc6381c9f1\": rpc error: code = NotFound desc = could not find container \"21ad9cf07eb792c0e8c1e3bdc3afb0a226d165a744991f5c50c88ccc6381c9f1\": container with ID starting with 21ad9cf07eb792c0e8c1e3bdc3afb0a226d165a744991f5c50c88ccc6381c9f1 not found: ID does not exist"
Oct 30 22:53:28.240698 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:53:28.240664    4412 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw1-7476d02544/isvc-paddle-v2-kserve-predictor-877ff9fb-c7btw"]
Oct 30 22:53:28.286324 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:53:28.286286    4412 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-gw1-7476d02544/isvc-paddle-v2-kserve-predictor-877ff9fb-c7btw"]
Oct 30 22:53:29.626861 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:53:29.626830    4412 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b2e98e5d-b30a-4277-b5c8-3086d229bea6" path="/var/lib/kubelet/pods/b2e98e5d-b30a-4277-b5c8-3086d229bea6/volumes"
Oct 30 22:53:32.738156 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:53:32.737770    4412 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-gw1-7476d02544/isvc-pmml-predictor-7b87459657-qqtht"]
Oct 30 22:53:32.738653 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:53:32.738544    4412 cpu_manager.go:414] "containerMap: removing container" podUID="b2e98e5d-b30a-4277-b5c8-3086d229bea6" containerName="kserve-container"
Oct 30 22:53:32.738653 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:53:32.738571    4412 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b2e98e5d-b30a-4277-b5c8-3086d229bea6" containerName="kserve-container"
Oct 30 22:53:32.738653 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:53:32.738601    4412 cpu_manager.go:414] "containerMap: removing container" podUID="1b060510-0ea0-46b8-b372-2f57e6857187" containerName="kserve-container"
Oct 30 22:53:32.738653 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:53:32.738609    4412 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1b060510-0ea0-46b8-b372-2f57e6857187" containerName="kserve-container"
Oct 30 22:53:32.738653 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:53:32.738636    4412 cpu_manager.go:414] "containerMap: removing container" podUID="1b060510-0ea0-46b8-b372-2f57e6857187" containerName="storage-initializer"
Oct 30 22:53:32.738653 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:53:32.738644    4412 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1b060510-0ea0-46b8-b372-2f57e6857187" containerName="storage-initializer"
Oct 30 22:53:32.738923 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:53:32.738667    4412 cpu_manager.go:414] "containerMap: removing container" podUID="b2e98e5d-b30a-4277-b5c8-3086d229bea6" containerName="kube-rbac-proxy"
Oct 30 22:53:32.738923 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:53:32.738675    4412 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b2e98e5d-b30a-4277-b5c8-3086d229bea6" containerName="kube-rbac-proxy"
Oct 30 22:53:32.738923 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:53:32.738687    4412 cpu_manager.go:414] "containerMap: removing container" podUID="1b060510-0ea0-46b8-b372-2f57e6857187" containerName="kube-rbac-proxy"
Oct 30 22:53:32.738923 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:53:32.738694    4412 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1b060510-0ea0-46b8-b372-2f57e6857187" containerName="kube-rbac-proxy"
Oct 30 22:53:32.738923 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:53:32.738726    4412 cpu_manager.go:414] "containerMap: removing container" podUID="b2e98e5d-b30a-4277-b5c8-3086d229bea6" containerName="storage-initializer"
Oct 30 22:53:32.738923 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:53:32.738737    4412 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b2e98e5d-b30a-4277-b5c8-3086d229bea6" containerName="storage-initializer"
Oct 30 22:53:32.738923 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:53:32.738859    4412 memory_manager.go:365] "RemoveStaleState removing state" podUID="b2e98e5d-b30a-4277-b5c8-3086d229bea6" containerName="kserve-container"
Oct 30 22:53:32.738923 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:53:32.738871    4412 memory_manager.go:365] "RemoveStaleState removing state" podUID="1b060510-0ea0-46b8-b372-2f57e6857187" containerName="kserve-container"
Oct 30 22:53:32.738923 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:53:32.738888    4412 memory_manager.go:365] "RemoveStaleState removing state" podUID="b2e98e5d-b30a-4277-b5c8-3086d229bea6" containerName="kube-rbac-proxy"
Oct 30 22:53:32.738923 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:53:32.738895    4412 memory_manager.go:365] "RemoveStaleState removing state" podUID="1b060510-0ea0-46b8-b372-2f57e6857187" containerName="kube-rbac-proxy"
Oct 30 22:53:32.744759 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:53:32.744740    4412 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-7476d02544/isvc-pmml-predictor-7b87459657-qqtht"
Oct 30 22:53:32.799017 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:53:32.798979    4412 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw1-7476d02544\"/\"odh-kserve-custom-ca-bundle\""
Oct 30 22:53:32.799220 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:53:32.799042    4412 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw1-7476d02544\"/\"seaweedfs-s3-creds\""
Oct 30 22:53:32.799220 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:53:32.799042    4412 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw1-7476d02544\"/\"isvc-pmml-kube-rbac-proxy-sar-config\""
Oct 30 22:53:32.849072 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:53:32.849044    4412 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw1-7476d02544\"/\"default-dockercfg-cxfkn\""
Oct 30 22:53:32.849072 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:53:32.849044    4412 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw1-7476d02544\"/\"openshift-service-ca.crt\""
Oct 30 22:53:32.849072 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:53:32.849074    4412 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw1-7476d02544\"/\"isvc-pmml-predictor-serving-cert\""
Oct 30 22:53:32.849396 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:53:32.849143    4412 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw1-7476d02544\"/\"kube-root-ca.crt\""
Oct 30 22:53:32.850485 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:53:32.850461    4412 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw1-7476d02544/isvc-pmml-predictor-7b87459657-qqtht"]
Oct 30 22:53:32.853593 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:53:32.853571    4412 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/aee7a946-1a22-45f0-b8ef-d6d46b99fb7b-kserve-provision-location\") pod \"isvc-pmml-predictor-7b87459657-qqtht\" (UID: \"aee7a946-1a22-45f0-b8ef-d6d46b99fb7b\") " pod="e2e-gw1-7476d02544/isvc-pmml-predictor-7b87459657-qqtht"
Oct 30 22:53:32.853708 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:53:32.853618    4412 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/aee7a946-1a22-45f0-b8ef-d6d46b99fb7b-kube-rbac-proxy-sar-config\") pod \"isvc-pmml-predictor-7b87459657-qqtht\" (UID: \"aee7a946-1a22-45f0-b8ef-d6d46b99fb7b\") " pod="e2e-gw1-7476d02544/isvc-pmml-predictor-7b87459657-qqtht"
Oct 30 22:53:32.853708 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:53:32.853682    4412 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/aee7a946-1a22-45f0-b8ef-d6d46b99fb7b-cabundle-cert\") pod \"isvc-pmml-predictor-7b87459657-qqtht\" (UID: \"aee7a946-1a22-45f0-b8ef-d6d46b99fb7b\") " pod="e2e-gw1-7476d02544/isvc-pmml-predictor-7b87459657-qqtht"
Oct 30 22:53:32.853836 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:53:32.853729    4412 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w4lfh\" (UniqueName: \"kubernetes.io/projected/aee7a946-1a22-45f0-b8ef-d6d46b99fb7b-kube-api-access-w4lfh\") pod \"isvc-pmml-predictor-7b87459657-qqtht\" (UID: \"aee7a946-1a22-45f0-b8ef-d6d46b99fb7b\") " pod="e2e-gw1-7476d02544/isvc-pmml-predictor-7b87459657-qqtht"
Oct 30 22:53:32.853836 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:53:32.853807    4412 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/aee7a946-1a22-45f0-b8ef-d6d46b99fb7b-proxy-tls\") pod \"isvc-pmml-predictor-7b87459657-qqtht\" (UID: \"aee7a946-1a22-45f0-b8ef-d6d46b99fb7b\") " pod="e2e-gw1-7476d02544/isvc-pmml-predictor-7b87459657-qqtht"
Oct 30 22:53:32.954716 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:53:32.954682    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/aee7a946-1a22-45f0-b8ef-d6d46b99fb7b-kube-rbac-proxy-sar-config\") pod \"isvc-pmml-predictor-7b87459657-qqtht\" (UID: \"aee7a946-1a22-45f0-b8ef-d6d46b99fb7b\") " pod="e2e-gw1-7476d02544/isvc-pmml-predictor-7b87459657-qqtht"
Oct 30 22:53:32.954716 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:53:32.954720    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/aee7a946-1a22-45f0-b8ef-d6d46b99fb7b-cabundle-cert\") pod \"isvc-pmml-predictor-7b87459657-qqtht\" (UID: \"aee7a946-1a22-45f0-b8ef-d6d46b99fb7b\") " pod="e2e-gw1-7476d02544/isvc-pmml-predictor-7b87459657-qqtht"
Oct 30 22:53:32.954955 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:53:32.954764    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w4lfh\" (UniqueName: \"kubernetes.io/projected/aee7a946-1a22-45f0-b8ef-d6d46b99fb7b-kube-api-access-w4lfh\") pod \"isvc-pmml-predictor-7b87459657-qqtht\" (UID: \"aee7a946-1a22-45f0-b8ef-d6d46b99fb7b\") " pod="e2e-gw1-7476d02544/isvc-pmml-predictor-7b87459657-qqtht"
Oct 30 22:53:32.954955 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:53:32.954789    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/aee7a946-1a22-45f0-b8ef-d6d46b99fb7b-proxy-tls\") pod \"isvc-pmml-predictor-7b87459657-qqtht\" (UID: \"aee7a946-1a22-45f0-b8ef-d6d46b99fb7b\") " pod="e2e-gw1-7476d02544/isvc-pmml-predictor-7b87459657-qqtht"
Oct 30 22:53:32.954955 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:53:32.954821    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/aee7a946-1a22-45f0-b8ef-d6d46b99fb7b-kserve-provision-location\") pod \"isvc-pmml-predictor-7b87459657-qqtht\" (UID: \"aee7a946-1a22-45f0-b8ef-d6d46b99fb7b\") " pod="e2e-gw1-7476d02544/isvc-pmml-predictor-7b87459657-qqtht"
Oct 30 22:53:32.955271 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:53:32.955251    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/aee7a946-1a22-45f0-b8ef-d6d46b99fb7b-kserve-provision-location\") pod \"isvc-pmml-predictor-7b87459657-qqtht\" (UID: \"aee7a946-1a22-45f0-b8ef-d6d46b99fb7b\") " pod="e2e-gw1-7476d02544/isvc-pmml-predictor-7b87459657-qqtht"
Oct 30 22:53:32.955544 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:53:32.955511    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/aee7a946-1a22-45f0-b8ef-d6d46b99fb7b-kube-rbac-proxy-sar-config\") pod \"isvc-pmml-predictor-7b87459657-qqtht\" (UID: \"aee7a946-1a22-45f0-b8ef-d6d46b99fb7b\") " pod="e2e-gw1-7476d02544/isvc-pmml-predictor-7b87459657-qqtht"
Oct 30 22:53:32.955654 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:53:32.955564    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/aee7a946-1a22-45f0-b8ef-d6d46b99fb7b-cabundle-cert\") pod \"isvc-pmml-predictor-7b87459657-qqtht\" (UID: \"aee7a946-1a22-45f0-b8ef-d6d46b99fb7b\") " pod="e2e-gw1-7476d02544/isvc-pmml-predictor-7b87459657-qqtht"
Oct 30 22:53:32.957533 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:53:32.957504    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/aee7a946-1a22-45f0-b8ef-d6d46b99fb7b-proxy-tls\") pod \"isvc-pmml-predictor-7b87459657-qqtht\" (UID: \"aee7a946-1a22-45f0-b8ef-d6d46b99fb7b\") " pod="e2e-gw1-7476d02544/isvc-pmml-predictor-7b87459657-qqtht"
Oct 30 22:53:33.014012 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:53:33.013939    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w4lfh\" (UniqueName: \"kubernetes.io/projected/aee7a946-1a22-45f0-b8ef-d6d46b99fb7b-kube-api-access-w4lfh\") pod \"isvc-pmml-predictor-7b87459657-qqtht\" (UID: \"aee7a946-1a22-45f0-b8ef-d6d46b99fb7b\") " pod="e2e-gw1-7476d02544/isvc-pmml-predictor-7b87459657-qqtht"
Oct 30 22:53:33.061533 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:53:33.061495    4412 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-7476d02544/isvc-pmml-predictor-7b87459657-qqtht"
Oct 30 22:53:33.481590 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:53:33.481563    4412 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw1-7476d02544/isvc-pmml-predictor-7b87459657-qqtht"]
Oct 30 22:53:33.484003 ip-10-0-131-202 kubenswrapper[4412]: W1030 22:53:33.483969    4412 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaee7a946_1a22_45f0_b8ef_d6d46b99fb7b.slice/crio-ba88665a5dd68f0a7d5fadc9864c8c986e9250870c5f5d8f7f41f1090a061a5e WatchSource:0}: Error finding container ba88665a5dd68f0a7d5fadc9864c8c986e9250870c5f5d8f7f41f1090a061a5e: Status 404 returned error can't find the container with id ba88665a5dd68f0a7d5fadc9864c8c986e9250870c5f5d8f7f41f1090a061a5e
Oct 30 22:53:34.100828 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:53:34.100791    4412 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-7476d02544/isvc-pmml-predictor-7b87459657-qqtht" event={"ID":"aee7a946-1a22-45f0-b8ef-d6d46b99fb7b","Type":"ContainerStarted","Data":"37c6c32228bb8006cc1ee321eb5b99d5454a5dd28b449ca5268528c23c249050"}
Oct 30 22:53:34.101297 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:53:34.100847    4412 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-7476d02544/isvc-pmml-predictor-7b87459657-qqtht" event={"ID":"aee7a946-1a22-45f0-b8ef-d6d46b99fb7b","Type":"ContainerStarted","Data":"ba88665a5dd68f0a7d5fadc9864c8c986e9250870c5f5d8f7f41f1090a061a5e"}
Oct 30 22:53:34.448959 ip-10-0-131-202 kubenswrapper[4412]: E1030 22:53:34.448866    4412 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bkhmz" containerName="extractor"
Oct 30 22:53:38.116458 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:53:38.116427    4412 generic.go:356] "Generic (PLEG): container finished" podID="aee7a946-1a22-45f0-b8ef-d6d46b99fb7b" containerID="37c6c32228bb8006cc1ee321eb5b99d5454a5dd28b449ca5268528c23c249050" exitCode=0
Oct 30 22:53:38.116854 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:53:38.116498    4412 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-7476d02544/isvc-pmml-predictor-7b87459657-qqtht" event={"ID":"aee7a946-1a22-45f0-b8ef-d6d46b99fb7b","Type":"ContainerDied","Data":"37c6c32228bb8006cc1ee321eb5b99d5454a5dd28b449ca5268528c23c249050"}
Oct 30 22:53:39.122474 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:53:39.122439    4412 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-7476d02544/isvc-pmml-predictor-7b87459657-qqtht" event={"ID":"aee7a946-1a22-45f0-b8ef-d6d46b99fb7b","Type":"ContainerStarted","Data":"b9471ff9006ecb96926e347f9ddc083faf6cbd1f9621740bdfff84bcfd56bbcd"}
Oct 30 22:53:39.122875 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:53:39.122503    4412 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-7476d02544/isvc-pmml-predictor-7b87459657-qqtht" event={"ID":"aee7a946-1a22-45f0-b8ef-d6d46b99fb7b","Type":"ContainerStarted","Data":"311c8f7ba377b7844f8d9368b283ac4d64237b43a8e2f46925ebeea1988c6a64"}
Oct 30 22:53:39.122875 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:53:39.122718    4412 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw1-7476d02544/isvc-pmml-predictor-7b87459657-qqtht"
Oct 30 22:53:39.246158 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:53:39.246085    4412 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-gw1-7476d02544/isvc-pmml-predictor-7b87459657-qqtht" podStartSLOduration=3.234174863 podStartE2EDuration="7.246071125s" totalImagesPullingTime="0s" totalInitContainerRuntime="4.011896262s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:53:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:53:39.245142144 +0000 UTC m=+3639.749814100" watchObservedRunningTime="2026-10-30 22:53:39.246071125 +0000 UTC m=+3639.750743074"
Oct 30 22:53:40.127037 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:53:40.127001    4412 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw1-7476d02544/isvc-pmml-predictor-7b87459657-qqtht"
Oct 30 22:53:40.128352 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:53:40.128314    4412 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-7476d02544/isvc-pmml-predictor-7b87459657-qqtht" podUID="aee7a946-1a22-45f0-b8ef-d6d46b99fb7b" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.39:8080: connect: connection refused"
Oct 30 22:53:41.130479 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:53:41.130437    4412 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-7476d02544/isvc-pmml-predictor-7b87459657-qqtht" podUID="aee7a946-1a22-45f0-b8ef-d6d46b99fb7b" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.39:8080: connect: connection refused"
Oct 30 22:53:46.135426 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:53:46.135393    4412 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw1-7476d02544/isvc-pmml-predictor-7b87459657-qqtht"
Oct 30 22:53:46.135924 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:53:46.135895    4412 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-7476d02544/isvc-pmml-predictor-7b87459657-qqtht" podUID="aee7a946-1a22-45f0-b8ef-d6d46b99fb7b" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.39:8080: connect: connection refused"
Oct 30 22:53:56.136580 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:53:56.136537    4412 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-7476d02544/isvc-pmml-predictor-7b87459657-qqtht" podUID="aee7a946-1a22-45f0-b8ef-d6d46b99fb7b" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.39:8080: connect: connection refused"
Oct 30 22:54:04.449304 ip-10-0-131-202 kubenswrapper[4412]: E1030 22:54:04.449199    4412 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bkhmz" containerName="extractor"
Oct 30 22:54:06.136250 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:54:06.136209    4412 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-7476d02544/isvc-pmml-predictor-7b87459657-qqtht" podUID="aee7a946-1a22-45f0-b8ef-d6d46b99fb7b" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.39:8080: connect: connection refused"
Oct 30 22:54:16.136803 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:54:16.136763    4412 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-7476d02544/isvc-pmml-predictor-7b87459657-qqtht" podUID="aee7a946-1a22-45f0-b8ef-d6d46b99fb7b" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.39:8080: connect: connection refused"
Oct 30 22:54:23.158208 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:54:23.158175    4412 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-gw0-37d062d1dd/isvc-sklearn-mcp-predictor-79545c9b5b-nmkbp"]
Oct 30 22:54:23.161936 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:54:23.161913    4412 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-37d062d1dd/isvc-sklearn-mcp-predictor-79545c9b5b-nmkbp"
Oct 30 22:54:23.215112 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:54:23.215077    4412 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw0-37d062d1dd\"/\"seaweedfs-s3-creds\""
Oct 30 22:54:23.252654 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:54:23.252620    4412 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/ca32745c-b63f-4e11-9881-2cbb64726053-cabundle-cert\") pod \"isvc-sklearn-mcp-predictor-79545c9b5b-nmkbp\" (UID: \"ca32745c-b63f-4e11-9881-2cbb64726053\") " pod="e2e-gw0-37d062d1dd/isvc-sklearn-mcp-predictor-79545c9b5b-nmkbp"
Oct 30 22:54:23.252776 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:54:23.252666    4412 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rx62n\" (UniqueName: \"kubernetes.io/projected/ca32745c-b63f-4e11-9881-2cbb64726053-kube-api-access-rx62n\") pod \"isvc-sklearn-mcp-predictor-79545c9b5b-nmkbp\" (UID: \"ca32745c-b63f-4e11-9881-2cbb64726053\") " pod="e2e-gw0-37d062d1dd/isvc-sklearn-mcp-predictor-79545c9b5b-nmkbp"
Oct 30 22:54:23.252776 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:54:23.252732    4412 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/ca32745c-b63f-4e11-9881-2cbb64726053-proxy-tls\") pod \"isvc-sklearn-mcp-predictor-79545c9b5b-nmkbp\" (UID: \"ca32745c-b63f-4e11-9881-2cbb64726053\") " pod="e2e-gw0-37d062d1dd/isvc-sklearn-mcp-predictor-79545c9b5b-nmkbp"
Oct 30 22:54:23.252776 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:54:23.252772    4412 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/ca32745c-b63f-4e11-9881-2cbb64726053-kserve-provision-location\") pod \"isvc-sklearn-mcp-predictor-79545c9b5b-nmkbp\" (UID: \"ca32745c-b63f-4e11-9881-2cbb64726053\") " pod="e2e-gw0-37d062d1dd/isvc-sklearn-mcp-predictor-79545c9b5b-nmkbp"
Oct 30 22:54:23.252894 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:54:23.252829    4412 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/ca32745c-b63f-4e11-9881-2cbb64726053-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-mcp-predictor-79545c9b5b-nmkbp\" (UID: \"ca32745c-b63f-4e11-9881-2cbb64726053\") " pod="e2e-gw0-37d062d1dd/isvc-sklearn-mcp-predictor-79545c9b5b-nmkbp"
Oct 30 22:54:23.265361 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:54:23.265328    4412 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw0-37d062d1dd\"/\"openshift-service-ca.crt\""
Oct 30 22:54:23.265361 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:54:23.265336    4412 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw0-37d062d1dd\"/\"isvc-sklearn-mcp-kube-rbac-proxy-sar-config\""
Oct 30 22:54:23.265533 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:54:23.265369    4412 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw0-37d062d1dd\"/\"kube-root-ca.crt\""
Oct 30 22:54:23.265533 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:54:23.265415    4412 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw0-37d062d1dd\"/\"odh-kserve-custom-ca-bundle\""
Oct 30 22:54:23.265533 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:54:23.265493    4412 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw0-37d062d1dd\"/\"isvc-sklearn-mcp-predictor-serving-cert\""
Oct 30 22:54:23.265715 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:54:23.265689    4412 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw0-37d062d1dd\"/\"default-dockercfg-4v4s8\""
Oct 30 22:54:23.266999 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:54:23.266976    4412 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw0-37d062d1dd/isvc-sklearn-mcp-predictor-79545c9b5b-nmkbp"]
Oct 30 22:54:23.353221 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:54:23.353190    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rx62n\" (UniqueName: \"kubernetes.io/projected/ca32745c-b63f-4e11-9881-2cbb64726053-kube-api-access-rx62n\") pod \"isvc-sklearn-mcp-predictor-79545c9b5b-nmkbp\" (UID: \"ca32745c-b63f-4e11-9881-2cbb64726053\") " pod="e2e-gw0-37d062d1dd/isvc-sklearn-mcp-predictor-79545c9b5b-nmkbp"
Oct 30 22:54:23.353221 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:54:23.353228    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/ca32745c-b63f-4e11-9881-2cbb64726053-proxy-tls\") pod \"isvc-sklearn-mcp-predictor-79545c9b5b-nmkbp\" (UID: \"ca32745c-b63f-4e11-9881-2cbb64726053\") " pod="e2e-gw0-37d062d1dd/isvc-sklearn-mcp-predictor-79545c9b5b-nmkbp"
Oct 30 22:54:23.353486 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:54:23.353275    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/ca32745c-b63f-4e11-9881-2cbb64726053-kserve-provision-location\") pod \"isvc-sklearn-mcp-predictor-79545c9b5b-nmkbp\" (UID: \"ca32745c-b63f-4e11-9881-2cbb64726053\") " pod="e2e-gw0-37d062d1dd/isvc-sklearn-mcp-predictor-79545c9b5b-nmkbp"
Oct 30 22:54:23.353486 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:54:23.353297    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/ca32745c-b63f-4e11-9881-2cbb64726053-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-mcp-predictor-79545c9b5b-nmkbp\" (UID: \"ca32745c-b63f-4e11-9881-2cbb64726053\") " pod="e2e-gw0-37d062d1dd/isvc-sklearn-mcp-predictor-79545c9b5b-nmkbp"
Oct 30 22:54:23.353486 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:54:23.353318    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/ca32745c-b63f-4e11-9881-2cbb64726053-cabundle-cert\") pod \"isvc-sklearn-mcp-predictor-79545c9b5b-nmkbp\" (UID: \"ca32745c-b63f-4e11-9881-2cbb64726053\") " pod="e2e-gw0-37d062d1dd/isvc-sklearn-mcp-predictor-79545c9b5b-nmkbp"
Oct 30 22:54:23.353732 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:54:23.353711    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/ca32745c-b63f-4e11-9881-2cbb64726053-kserve-provision-location\") pod \"isvc-sklearn-mcp-predictor-79545c9b5b-nmkbp\" (UID: \"ca32745c-b63f-4e11-9881-2cbb64726053\") " pod="e2e-gw0-37d062d1dd/isvc-sklearn-mcp-predictor-79545c9b5b-nmkbp"
Oct 30 22:54:23.354019 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:54:23.353989    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/ca32745c-b63f-4e11-9881-2cbb64726053-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-mcp-predictor-79545c9b5b-nmkbp\" (UID: \"ca32745c-b63f-4e11-9881-2cbb64726053\") " pod="e2e-gw0-37d062d1dd/isvc-sklearn-mcp-predictor-79545c9b5b-nmkbp"
Oct 30 22:54:23.354170 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:54:23.354048    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/ca32745c-b63f-4e11-9881-2cbb64726053-cabundle-cert\") pod \"isvc-sklearn-mcp-predictor-79545c9b5b-nmkbp\" (UID: \"ca32745c-b63f-4e11-9881-2cbb64726053\") " pod="e2e-gw0-37d062d1dd/isvc-sklearn-mcp-predictor-79545c9b5b-nmkbp"
Oct 30 22:54:23.355926 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:54:23.355912    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/ca32745c-b63f-4e11-9881-2cbb64726053-proxy-tls\") pod \"isvc-sklearn-mcp-predictor-79545c9b5b-nmkbp\" (UID: \"ca32745c-b63f-4e11-9881-2cbb64726053\") " pod="e2e-gw0-37d062d1dd/isvc-sklearn-mcp-predictor-79545c9b5b-nmkbp"
Oct 30 22:54:23.411328 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:54:23.411256    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rx62n\" (UniqueName: \"kubernetes.io/projected/ca32745c-b63f-4e11-9881-2cbb64726053-kube-api-access-rx62n\") pod \"isvc-sklearn-mcp-predictor-79545c9b5b-nmkbp\" (UID: \"ca32745c-b63f-4e11-9881-2cbb64726053\") " pod="e2e-gw0-37d062d1dd/isvc-sklearn-mcp-predictor-79545c9b5b-nmkbp"
Oct 30 22:54:23.482316 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:54:23.482276    4412 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-37d062d1dd/isvc-sklearn-mcp-predictor-79545c9b5b-nmkbp"
Oct 30 22:54:23.905069 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:54:23.905045    4412 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw0-37d062d1dd/isvc-sklearn-mcp-predictor-79545c9b5b-nmkbp"]
Oct 30 22:54:23.907251 ip-10-0-131-202 kubenswrapper[4412]: W1030 22:54:23.907226    4412 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podca32745c_b63f_4e11_9881_2cbb64726053.slice/crio-b3da522c7dd205eac60c7e306f60ca851abddaa2e53b450f6f5f1d4a81ddb9a7 WatchSource:0}: Error finding container b3da522c7dd205eac60c7e306f60ca851abddaa2e53b450f6f5f1d4a81ddb9a7: Status 404 returned error can't find the container with id b3da522c7dd205eac60c7e306f60ca851abddaa2e53b450f6f5f1d4a81ddb9a7
Oct 30 22:54:24.293656 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:54:24.293622    4412 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-37d062d1dd/isvc-sklearn-mcp-predictor-79545c9b5b-nmkbp" event={"ID":"ca32745c-b63f-4e11-9881-2cbb64726053","Type":"ContainerStarted","Data":"f381b7850ab6052d3b9c16e572191d3da1a7ca3a288333a3b97f1347cc3edc3b"}
Oct 30 22:54:24.294037 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:54:24.293685    4412 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-37d062d1dd/isvc-sklearn-mcp-predictor-79545c9b5b-nmkbp" event={"ID":"ca32745c-b63f-4e11-9881-2cbb64726053","Type":"ContainerStarted","Data":"b3da522c7dd205eac60c7e306f60ca851abddaa2e53b450f6f5f1d4a81ddb9a7"}
Oct 30 22:54:26.136761 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:54:26.136726    4412 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-7476d02544/isvc-pmml-predictor-7b87459657-qqtht" podUID="aee7a946-1a22-45f0-b8ef-d6d46b99fb7b" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.39:8080: connect: connection refused"
Oct 30 22:54:28.309343 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:54:28.309310    4412 generic.go:356] "Generic (PLEG): container finished" podID="ca32745c-b63f-4e11-9881-2cbb64726053" containerID="f381b7850ab6052d3b9c16e572191d3da1a7ca3a288333a3b97f1347cc3edc3b" exitCode=0
Oct 30 22:54:28.309767 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:54:28.309370    4412 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-37d062d1dd/isvc-sklearn-mcp-predictor-79545c9b5b-nmkbp" event={"ID":"ca32745c-b63f-4e11-9881-2cbb64726053","Type":"ContainerDied","Data":"f381b7850ab6052d3b9c16e572191d3da1a7ca3a288333a3b97f1347cc3edc3b"}
Oct 30 22:54:29.315318 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:54:29.315283    4412 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-37d062d1dd/isvc-sklearn-mcp-predictor-79545c9b5b-nmkbp" event={"ID":"ca32745c-b63f-4e11-9881-2cbb64726053","Type":"ContainerStarted","Data":"699469970ab06bd63b82cd7e8d6f6615f08ecf2ba57443d32e2c7276e34fbca6"}
Oct 30 22:54:34.453572 ip-10-0-131-202 kubenswrapper[4412]: E1030 22:54:34.453487    4412 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bkhmz" containerName="extractor"
Oct 30 22:54:35.345622 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:54:35.345588    4412 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-37d062d1dd/isvc-sklearn-mcp-predictor-79545c9b5b-nmkbp" event={"ID":"ca32745c-b63f-4e11-9881-2cbb64726053","Type":"ContainerStarted","Data":"e3890b8f0a9d49910f26ea7d8a97f533eada1c5e0ebe195ddcf3e0a02fbe4ccc"}
Oct 30 22:54:35.345809 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:54:35.345651    4412 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-37d062d1dd/isvc-sklearn-mcp-predictor-79545c9b5b-nmkbp" event={"ID":"ca32745c-b63f-4e11-9881-2cbb64726053","Type":"ContainerStarted","Data":"37525ef5950a616805dbbbe8c07dd9fa20952f07652ee7324c0dc4851871a0c1"}
Oct 30 22:54:35.345809 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:54:35.345781    4412 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw0-37d062d1dd/isvc-sklearn-mcp-predictor-79545c9b5b-nmkbp"
Oct 30 22:54:35.567485 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:54:35.567434    4412 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-gw0-37d062d1dd/isvc-sklearn-mcp-predictor-79545c9b5b-nmkbp" podStartSLOduration=2.28517199 podStartE2EDuration="12.567420958s" totalImagesPullingTime="6.422318631s" totalInitContainerRuntime="3.859930337s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:54:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:54:35.516850154 +0000 UTC m=+3696.021522105" watchObservedRunningTime="2026-10-30 22:54:35.567420958 +0000 UTC m=+3696.072092910"
Oct 30 22:54:36.136237 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:54:36.136200    4412 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-7476d02544/isvc-pmml-predictor-7b87459657-qqtht" podUID="aee7a946-1a22-45f0-b8ef-d6d46b99fb7b" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.39:8080: connect: connection refused"
Oct 30 22:54:36.350340 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:54:36.350312    4412 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw0-37d062d1dd/isvc-sklearn-mcp-predictor-79545c9b5b-nmkbp"
Oct 30 22:54:36.350589 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:54:36.350577    4412 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw0-37d062d1dd/isvc-sklearn-mcp-predictor-79545c9b5b-nmkbp"
Oct 30 22:54:42.360392 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:54:42.360365    4412 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw0-37d062d1dd/isvc-sklearn-mcp-predictor-79545c9b5b-nmkbp"
Oct 30 22:54:46.136789 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:54:46.136749    4412 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-7476d02544/isvc-pmml-predictor-7b87459657-qqtht" podUID="aee7a946-1a22-45f0-b8ef-d6d46b99fb7b" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.39:8080: connect: connection refused"
Oct 30 22:54:56.136867 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:54:56.136825    4412 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-7476d02544/isvc-pmml-predictor-7b87459657-qqtht" podUID="aee7a946-1a22-45f0-b8ef-d6d46b99fb7b" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.39:8080: connect: connection refused"
Oct 30 22:55:02.361630 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:55:02.361594    4412 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-37d062d1dd/isvc-sklearn-mcp-predictor-79545c9b5b-nmkbp" podUID="ca32745c-b63f-4e11-9881-2cbb64726053" containerName="kserve-container" probeResult="failure" output="Get \"http://10.133.0.40:8080/v1/models/isvc-sklearn-mcp\": dial tcp 10.133.0.40:8080: connect: connection refused"
Oct 30 22:55:03.483332 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:55:03.483292    4412 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-gw0-37d062d1dd/isvc-sklearn-mcp-predictor-79545c9b5b-nmkbp" podUID="ca32745c-b63f-4e11-9881-2cbb64726053" containerName="kserve-container" probeResult="failure" output="Get \"http://10.133.0.40:8080/v1/models/isvc-sklearn-mcp\": dial tcp 10.133.0.40:8080: connect: connection refused"
Oct 30 22:55:04.450042 ip-10-0-131-202 kubenswrapper[4412]: E1030 22:55:04.450004    4412 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bkhmz" containerName="extractor"
Oct 30 22:55:04.621924 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:55:04.621883    4412 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-7476d02544/isvc-pmml-predictor-7b87459657-qqtht" podUID="aee7a946-1a22-45f0-b8ef-d6d46b99fb7b" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.39:8080: connect: connection refused"
Oct 30 22:55:12.546925 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:55:12.546832    4412 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw0-37d062d1dd/isvc-sklearn-mcp-predictor-79545c9b5b-nmkbp"
Oct 30 22:55:14.622813 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:55:14.622777    4412 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw1-7476d02544/isvc-pmml-predictor-7b87459657-qqtht"
Oct 30 22:55:23.819212 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:55:23.819172    4412 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw1-7476d02544/isvc-pmml-predictor-7b87459657-qqtht"]
Oct 30 22:55:23.819602 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:55:23.819409    4412 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw1-7476d02544/isvc-pmml-predictor-7b87459657-qqtht" podUID="aee7a946-1a22-45f0-b8ef-d6d46b99fb7b" containerName="kserve-container" containerID="cri-o://311c8f7ba377b7844f8d9368b283ac4d64237b43a8e2f46925ebeea1988c6a64" gracePeriod=30
Oct 30 22:55:23.819602 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:55:23.819440    4412 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw1-7476d02544/isvc-pmml-predictor-7b87459657-qqtht" podUID="aee7a946-1a22-45f0-b8ef-d6d46b99fb7b" containerName="kube-rbac-proxy" containerID="cri-o://b9471ff9006ecb96926e347f9ddc083faf6cbd1f9621740bdfff84bcfd56bbcd" gracePeriod=30
Oct 30 22:55:24.531386 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:55:24.531354    4412 generic.go:356] "Generic (PLEG): container finished" podID="aee7a946-1a22-45f0-b8ef-d6d46b99fb7b" containerID="b9471ff9006ecb96926e347f9ddc083faf6cbd1f9621740bdfff84bcfd56bbcd" exitCode=0
Oct 30 22:55:24.531569 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:55:24.531428    4412 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-7476d02544/isvc-pmml-predictor-7b87459657-qqtht" event={"ID":"aee7a946-1a22-45f0-b8ef-d6d46b99fb7b","Type":"ContainerDied","Data":"b9471ff9006ecb96926e347f9ddc083faf6cbd1f9621740bdfff84bcfd56bbcd"}
Oct 30 22:55:24.621908 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:55:24.621870    4412 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-7476d02544/isvc-pmml-predictor-7b87459657-qqtht" podUID="aee7a946-1a22-45f0-b8ef-d6d46b99fb7b" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.39:8080: connect: connection refused"
Oct 30 22:55:26.131594 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:55:26.131553    4412 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-7476d02544/isvc-pmml-predictor-7b87459657-qqtht" podUID="aee7a946-1a22-45f0-b8ef-d6d46b99fb7b" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.39:8643/healthz\": dial tcp 10.133.0.39:8643: connect: connection refused"
Oct 30 22:55:28.158905 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:55:28.158883    4412 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-7476d02544/isvc-pmml-predictor-7b87459657-qqtht"
Oct 30 22:55:28.299611 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:55:28.299577    4412 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/aee7a946-1a22-45f0-b8ef-d6d46b99fb7b-cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/aee7a946-1a22-45f0-b8ef-d6d46b99fb7b-cabundle-cert\") pod \"aee7a946-1a22-45f0-b8ef-d6d46b99fb7b\" (UID: \"aee7a946-1a22-45f0-b8ef-d6d46b99fb7b\") "
Oct 30 22:55:28.299791 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:55:28.299645    4412 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/aee7a946-1a22-45f0-b8ef-d6d46b99fb7b-kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/aee7a946-1a22-45f0-b8ef-d6d46b99fb7b-kserve-provision-location\") pod \"aee7a946-1a22-45f0-b8ef-d6d46b99fb7b\" (UID: \"aee7a946-1a22-45f0-b8ef-d6d46b99fb7b\") "
Oct 30 22:55:28.299791 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:55:28.299663    4412 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/aee7a946-1a22-45f0-b8ef-d6d46b99fb7b-kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/aee7a946-1a22-45f0-b8ef-d6d46b99fb7b-kube-rbac-proxy-sar-config\") pod \"aee7a946-1a22-45f0-b8ef-d6d46b99fb7b\" (UID: \"aee7a946-1a22-45f0-b8ef-d6d46b99fb7b\") "
Oct 30 22:55:28.299791 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:55:28.299763    4412 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/aee7a946-1a22-45f0-b8ef-d6d46b99fb7b-kube-api-access-w4lfh\" (UniqueName: \"kubernetes.io/projected/aee7a946-1a22-45f0-b8ef-d6d46b99fb7b-kube-api-access-w4lfh\") pod \"aee7a946-1a22-45f0-b8ef-d6d46b99fb7b\" (UID: \"aee7a946-1a22-45f0-b8ef-d6d46b99fb7b\") "
Oct 30 22:55:28.299967 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:55:28.299824    4412 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/aee7a946-1a22-45f0-b8ef-d6d46b99fb7b-proxy-tls\" (UniqueName: \"kubernetes.io/secret/aee7a946-1a22-45f0-b8ef-d6d46b99fb7b-proxy-tls\") pod \"aee7a946-1a22-45f0-b8ef-d6d46b99fb7b\" (UID: \"aee7a946-1a22-45f0-b8ef-d6d46b99fb7b\") "
Oct 30 22:55:28.300035 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:55:28.299968    4412 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aee7a946-1a22-45f0-b8ef-d6d46b99fb7b-kserve-provision-location" pod "aee7a946-1a22-45f0-b8ef-d6d46b99fb7b" (UID: "aee7a946-1a22-45f0-b8ef-d6d46b99fb7b"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:55:28.300035 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:55:28.299968    4412 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/aee7a946-1a22-45f0-b8ef-d6d46b99fb7b-cabundle-cert" pod "aee7a946-1a22-45f0-b8ef-d6d46b99fb7b" (UID: "aee7a946-1a22-45f0-b8ef-d6d46b99fb7b"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:55:28.300035 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:55:28.299994    4412 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/aee7a946-1a22-45f0-b8ef-d6d46b99fb7b-kube-rbac-proxy-sar-config" pod "aee7a946-1a22-45f0-b8ef-d6d46b99fb7b" (UID: "aee7a946-1a22-45f0-b8ef-d6d46b99fb7b"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:55:28.300223 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:55:28.300152    4412 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/aee7a946-1a22-45f0-b8ef-d6d46b99fb7b-cabundle-cert\") on node \"ip-10-0-131-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:55:28.300223 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:55:28.300167    4412 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/aee7a946-1a22-45f0-b8ef-d6d46b99fb7b-kserve-provision-location\") on node \"ip-10-0-131-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:55:28.300223 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:55:28.300178    4412 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/aee7a946-1a22-45f0-b8ef-d6d46b99fb7b-kube-rbac-proxy-sar-config\") on node \"ip-10-0-131-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:55:28.302137 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:55:28.302102    4412 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aee7a946-1a22-45f0-b8ef-d6d46b99fb7b-kube-api-access-w4lfh" pod "aee7a946-1a22-45f0-b8ef-d6d46b99fb7b" (UID: "aee7a946-1a22-45f0-b8ef-d6d46b99fb7b"). InnerVolumeSpecName "kube-api-access-w4lfh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:55:28.302230 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:55:28.302171    4412 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/aee7a946-1a22-45f0-b8ef-d6d46b99fb7b-proxy-tls" pod "aee7a946-1a22-45f0-b8ef-d6d46b99fb7b" (UID: "aee7a946-1a22-45f0-b8ef-d6d46b99fb7b"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:55:28.401183 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:55:28.401148    4412 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/aee7a946-1a22-45f0-b8ef-d6d46b99fb7b-proxy-tls\") on node \"ip-10-0-131-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:55:28.401183 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:55:28.401177    4412 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w4lfh\" (UniqueName: \"kubernetes.io/projected/aee7a946-1a22-45f0-b8ef-d6d46b99fb7b-kube-api-access-w4lfh\") on node \"ip-10-0-131-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:55:28.555938 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:55:28.555841    4412 generic.go:356] "Generic (PLEG): container finished" podID="aee7a946-1a22-45f0-b8ef-d6d46b99fb7b" containerID="311c8f7ba377b7844f8d9368b283ac4d64237b43a8e2f46925ebeea1988c6a64" exitCode=0
Oct 30 22:55:28.555938 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:55:28.555877    4412 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-7476d02544/isvc-pmml-predictor-7b87459657-qqtht" event={"ID":"aee7a946-1a22-45f0-b8ef-d6d46b99fb7b","Type":"ContainerDied","Data":"311c8f7ba377b7844f8d9368b283ac4d64237b43a8e2f46925ebeea1988c6a64"}
Oct 30 22:55:28.555938 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:55:28.555926    4412 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-7476d02544/isvc-pmml-predictor-7b87459657-qqtht" event={"ID":"aee7a946-1a22-45f0-b8ef-d6d46b99fb7b","Type":"ContainerDied","Data":"ba88665a5dd68f0a7d5fadc9864c8c986e9250870c5f5d8f7f41f1090a061a5e"}
Oct 30 22:55:28.555938 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:55:28.555931    4412 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-7476d02544/isvc-pmml-predictor-7b87459657-qqtht"
Oct 30 22:55:28.556290 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:55:28.555953    4412 scope.go:122] "RemoveContainer" containerID="b9471ff9006ecb96926e347f9ddc083faf6cbd1f9621740bdfff84bcfd56bbcd"
Oct 30 22:55:28.565526 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:55:28.565507    4412 scope.go:122] "RemoveContainer" containerID="311c8f7ba377b7844f8d9368b283ac4d64237b43a8e2f46925ebeea1988c6a64"
Oct 30 22:55:28.573147 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:55:28.573107    4412 scope.go:122] "RemoveContainer" containerID="37c6c32228bb8006cc1ee321eb5b99d5454a5dd28b449ca5268528c23c249050"
Oct 30 22:55:28.582785 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:55:28.582763    4412 scope.go:122] "RemoveContainer" containerID="b9471ff9006ecb96926e347f9ddc083faf6cbd1f9621740bdfff84bcfd56bbcd"
Oct 30 22:55:28.583419 ip-10-0-131-202 kubenswrapper[4412]: E1030 22:55:28.583396    4412 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b9471ff9006ecb96926e347f9ddc083faf6cbd1f9621740bdfff84bcfd56bbcd\": container with ID starting with b9471ff9006ecb96926e347f9ddc083faf6cbd1f9621740bdfff84bcfd56bbcd not found: ID does not exist" containerID="b9471ff9006ecb96926e347f9ddc083faf6cbd1f9621740bdfff84bcfd56bbcd"
Oct 30 22:55:28.583500 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:55:28.583429    4412 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b9471ff9006ecb96926e347f9ddc083faf6cbd1f9621740bdfff84bcfd56bbcd"} err="failed to get container status \"b9471ff9006ecb96926e347f9ddc083faf6cbd1f9621740bdfff84bcfd56bbcd\": rpc error: code = NotFound desc = could not find container \"b9471ff9006ecb96926e347f9ddc083faf6cbd1f9621740bdfff84bcfd56bbcd\": container with ID starting with b9471ff9006ecb96926e347f9ddc083faf6cbd1f9621740bdfff84bcfd56bbcd not found: ID does not exist"
Oct 30 22:55:28.583500 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:55:28.583448    4412 scope.go:122] "RemoveContainer" containerID="311c8f7ba377b7844f8d9368b283ac4d64237b43a8e2f46925ebeea1988c6a64"
Oct 30 22:55:28.583697 ip-10-0-131-202 kubenswrapper[4412]: E1030 22:55:28.583682    4412 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"311c8f7ba377b7844f8d9368b283ac4d64237b43a8e2f46925ebeea1988c6a64\": container with ID starting with 311c8f7ba377b7844f8d9368b283ac4d64237b43a8e2f46925ebeea1988c6a64 not found: ID does not exist" containerID="311c8f7ba377b7844f8d9368b283ac4d64237b43a8e2f46925ebeea1988c6a64"
Oct 30 22:55:28.583751 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:55:28.583701    4412 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"311c8f7ba377b7844f8d9368b283ac4d64237b43a8e2f46925ebeea1988c6a64"} err="failed to get container status \"311c8f7ba377b7844f8d9368b283ac4d64237b43a8e2f46925ebeea1988c6a64\": rpc error: code = NotFound desc = could not find container \"311c8f7ba377b7844f8d9368b283ac4d64237b43a8e2f46925ebeea1988c6a64\": container with ID starting with 311c8f7ba377b7844f8d9368b283ac4d64237b43a8e2f46925ebeea1988c6a64 not found: ID does not exist"
Oct 30 22:55:28.583751 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:55:28.583713    4412 scope.go:122] "RemoveContainer" containerID="37c6c32228bb8006cc1ee321eb5b99d5454a5dd28b449ca5268528c23c249050"
Oct 30 22:55:28.583942 ip-10-0-131-202 kubenswrapper[4412]: E1030 22:55:28.583924    4412 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"37c6c32228bb8006cc1ee321eb5b99d5454a5dd28b449ca5268528c23c249050\": container with ID starting with 37c6c32228bb8006cc1ee321eb5b99d5454a5dd28b449ca5268528c23c249050 not found: ID does not exist" containerID="37c6c32228bb8006cc1ee321eb5b99d5454a5dd28b449ca5268528c23c249050"
Oct 30 22:55:28.583985 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:55:28.583957    4412 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"37c6c32228bb8006cc1ee321eb5b99d5454a5dd28b449ca5268528c23c249050"} err="failed to get container status \"37c6c32228bb8006cc1ee321eb5b99d5454a5dd28b449ca5268528c23c249050\": rpc error: code = NotFound desc = could not find container \"37c6c32228bb8006cc1ee321eb5b99d5454a5dd28b449ca5268528c23c249050\": container with ID starting with 37c6c32228bb8006cc1ee321eb5b99d5454a5dd28b449ca5268528c23c249050 not found: ID does not exist"
Oct 30 22:55:28.722046 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:55:28.722012    4412 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw1-7476d02544/isvc-pmml-predictor-7b87459657-qqtht"]
Oct 30 22:55:28.723973 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:55:28.723945    4412 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-gw1-7476d02544/isvc-pmml-predictor-7b87459657-qqtht"]
Oct 30 22:55:29.625771 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:55:29.625743    4412 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="aee7a946-1a22-45f0-b8ef-d6d46b99fb7b" path="/var/lib/kubelet/pods/aee7a946-1a22-45f0-b8ef-d6d46b99fb7b/volumes"
Oct 30 22:55:30.808190 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:55:30.808158    4412 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-gw1-7476d02544/isvc-pmml-runtime-predictor-78c49cb5ff-h4r8f"]
Oct 30 22:55:30.808613 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:55:30.808460    4412 cpu_manager.go:414] "containerMap: removing container" podUID="aee7a946-1a22-45f0-b8ef-d6d46b99fb7b" containerName="kube-rbac-proxy"
Oct 30 22:55:30.808613 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:55:30.808470    4412 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="aee7a946-1a22-45f0-b8ef-d6d46b99fb7b" containerName="kube-rbac-proxy"
Oct 30 22:55:30.808613 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:55:30.808482    4412 cpu_manager.go:414] "containerMap: removing container" podUID="aee7a946-1a22-45f0-b8ef-d6d46b99fb7b" containerName="kserve-container"
Oct 30 22:55:30.808613 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:55:30.808486    4412 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="aee7a946-1a22-45f0-b8ef-d6d46b99fb7b" containerName="kserve-container"
Oct 30 22:55:30.808613 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:55:30.808508    4412 cpu_manager.go:414] "containerMap: removing container" podUID="aee7a946-1a22-45f0-b8ef-d6d46b99fb7b" containerName="storage-initializer"
Oct 30 22:55:30.808613 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:55:30.808515    4412 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="aee7a946-1a22-45f0-b8ef-d6d46b99fb7b" containerName="storage-initializer"
Oct 30 22:55:30.808613 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:55:30.808567    4412 memory_manager.go:365] "RemoveStaleState removing state" podUID="aee7a946-1a22-45f0-b8ef-d6d46b99fb7b" containerName="kube-rbac-proxy"
Oct 30 22:55:30.808613 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:55:30.808574    4412 memory_manager.go:365] "RemoveStaleState removing state" podUID="aee7a946-1a22-45f0-b8ef-d6d46b99fb7b" containerName="kserve-container"
Oct 30 22:55:30.813247 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:55:30.813228    4412 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-7476d02544/isvc-pmml-runtime-predictor-78c49cb5ff-h4r8f"
Oct 30 22:55:30.866991 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:55:30.866962    4412 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw1-7476d02544\"/\"odh-kserve-custom-ca-bundle\""
Oct 30 22:55:30.866991 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:55:30.866973    4412 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw1-7476d02544\"/\"isvc-pmml-runtime-kube-rbac-proxy-sar-config\""
Oct 30 22:55:30.867210 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:55:30.867026    4412 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw1-7476d02544\"/\"kube-root-ca.crt\""
Oct 30 22:55:30.917222 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:55:30.917106    4412 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw1-7476d02544\"/\"openshift-service-ca.crt\""
Oct 30 22:55:30.917222 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:55:30.917191    4412 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw1-7476d02544\"/\"seaweedfs-s3-creds\""
Oct 30 22:55:30.917488 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:55:30.917257    4412 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw1-7476d02544\"/\"default-dockercfg-cxfkn\""
Oct 30 22:55:30.917488 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:55:30.917277    4412 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw1-7476d02544\"/\"isvc-pmml-runtime-predictor-serving-cert\""
Oct 30 22:55:30.918838 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:55:30.918817    4412 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw1-7476d02544/isvc-pmml-runtime-predictor-78c49cb5ff-h4r8f"]
Oct 30 22:55:30.922095 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:55:30.922073    4412 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-569vf\" (UniqueName: \"kubernetes.io/projected/8eedbf1b-b2f9-4ee0-a610-3dcfeec0d5e6-kube-api-access-569vf\") pod \"isvc-pmml-runtime-predictor-78c49cb5ff-h4r8f\" (UID: \"8eedbf1b-b2f9-4ee0-a610-3dcfeec0d5e6\") " pod="e2e-gw1-7476d02544/isvc-pmml-runtime-predictor-78c49cb5ff-h4r8f"
Oct 30 22:55:30.922196 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:55:30.922150    4412 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/8eedbf1b-b2f9-4ee0-a610-3dcfeec0d5e6-kube-rbac-proxy-sar-config\") pod \"isvc-pmml-runtime-predictor-78c49cb5ff-h4r8f\" (UID: \"8eedbf1b-b2f9-4ee0-a610-3dcfeec0d5e6\") " pod="e2e-gw1-7476d02544/isvc-pmml-runtime-predictor-78c49cb5ff-h4r8f"
Oct 30 22:55:30.922196 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:55:30.922182    4412 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/8eedbf1b-b2f9-4ee0-a610-3dcfeec0d5e6-cabundle-cert\") pod \"isvc-pmml-runtime-predictor-78c49cb5ff-h4r8f\" (UID: \"8eedbf1b-b2f9-4ee0-a610-3dcfeec0d5e6\") " pod="e2e-gw1-7476d02544/isvc-pmml-runtime-predictor-78c49cb5ff-h4r8f"
Oct 30 22:55:30.922267 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:55:30.922226    4412 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/8eedbf1b-b2f9-4ee0-a610-3dcfeec0d5e6-proxy-tls\") pod \"isvc-pmml-runtime-predictor-78c49cb5ff-h4r8f\" (UID: \"8eedbf1b-b2f9-4ee0-a610-3dcfeec0d5e6\") " pod="e2e-gw1-7476d02544/isvc-pmml-runtime-predictor-78c49cb5ff-h4r8f"
Oct 30 22:55:30.922364 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:55:30.922338    4412 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/8eedbf1b-b2f9-4ee0-a610-3dcfeec0d5e6-kserve-provision-location\") pod \"isvc-pmml-runtime-predictor-78c49cb5ff-h4r8f\" (UID: \"8eedbf1b-b2f9-4ee0-a610-3dcfeec0d5e6\") " pod="e2e-gw1-7476d02544/isvc-pmml-runtime-predictor-78c49cb5ff-h4r8f"
Oct 30 22:55:31.023049 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:55:31.023009    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-569vf\" (UniqueName: \"kubernetes.io/projected/8eedbf1b-b2f9-4ee0-a610-3dcfeec0d5e6-kube-api-access-569vf\") pod \"isvc-pmml-runtime-predictor-78c49cb5ff-h4r8f\" (UID: \"8eedbf1b-b2f9-4ee0-a610-3dcfeec0d5e6\") " pod="e2e-gw1-7476d02544/isvc-pmml-runtime-predictor-78c49cb5ff-h4r8f"
Oct 30 22:55:31.023255 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:55:31.023061    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/8eedbf1b-b2f9-4ee0-a610-3dcfeec0d5e6-kube-rbac-proxy-sar-config\") pod \"isvc-pmml-runtime-predictor-78c49cb5ff-h4r8f\" (UID: \"8eedbf1b-b2f9-4ee0-a610-3dcfeec0d5e6\") " pod="e2e-gw1-7476d02544/isvc-pmml-runtime-predictor-78c49cb5ff-h4r8f"
Oct 30 22:55:31.023255 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:55:31.023189    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/8eedbf1b-b2f9-4ee0-a610-3dcfeec0d5e6-cabundle-cert\") pod \"isvc-pmml-runtime-predictor-78c49cb5ff-h4r8f\" (UID: \"8eedbf1b-b2f9-4ee0-a610-3dcfeec0d5e6\") " pod="e2e-gw1-7476d02544/isvc-pmml-runtime-predictor-78c49cb5ff-h4r8f"
Oct 30 22:55:31.023255 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:55:31.023233    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/8eedbf1b-b2f9-4ee0-a610-3dcfeec0d5e6-proxy-tls\") pod \"isvc-pmml-runtime-predictor-78c49cb5ff-h4r8f\" (UID: \"8eedbf1b-b2f9-4ee0-a610-3dcfeec0d5e6\") " pod="e2e-gw1-7476d02544/isvc-pmml-runtime-predictor-78c49cb5ff-h4r8f"
Oct 30 22:55:31.023369 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:55:31.023313    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/8eedbf1b-b2f9-4ee0-a610-3dcfeec0d5e6-kserve-provision-location\") pod \"isvc-pmml-runtime-predictor-78c49cb5ff-h4r8f\" (UID: \"8eedbf1b-b2f9-4ee0-a610-3dcfeec0d5e6\") " pod="e2e-gw1-7476d02544/isvc-pmml-runtime-predictor-78c49cb5ff-h4r8f"
Oct 30 22:55:31.023723 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:55:31.023703    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/8eedbf1b-b2f9-4ee0-a610-3dcfeec0d5e6-kserve-provision-location\") pod \"isvc-pmml-runtime-predictor-78c49cb5ff-h4r8f\" (UID: \"8eedbf1b-b2f9-4ee0-a610-3dcfeec0d5e6\") " pod="e2e-gw1-7476d02544/isvc-pmml-runtime-predictor-78c49cb5ff-h4r8f"
Oct 30 22:55:31.023861 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:55:31.023842    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/8eedbf1b-b2f9-4ee0-a610-3dcfeec0d5e6-cabundle-cert\") pod \"isvc-pmml-runtime-predictor-78c49cb5ff-h4r8f\" (UID: \"8eedbf1b-b2f9-4ee0-a610-3dcfeec0d5e6\") " pod="e2e-gw1-7476d02544/isvc-pmml-runtime-predictor-78c49cb5ff-h4r8f"
Oct 30 22:55:31.023900 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:55:31.023842    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/8eedbf1b-b2f9-4ee0-a610-3dcfeec0d5e6-kube-rbac-proxy-sar-config\") pod \"isvc-pmml-runtime-predictor-78c49cb5ff-h4r8f\" (UID: \"8eedbf1b-b2f9-4ee0-a610-3dcfeec0d5e6\") " pod="e2e-gw1-7476d02544/isvc-pmml-runtime-predictor-78c49cb5ff-h4r8f"
Oct 30 22:55:31.025972 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:55:31.025956    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/8eedbf1b-b2f9-4ee0-a610-3dcfeec0d5e6-proxy-tls\") pod \"isvc-pmml-runtime-predictor-78c49cb5ff-h4r8f\" (UID: \"8eedbf1b-b2f9-4ee0-a610-3dcfeec0d5e6\") " pod="e2e-gw1-7476d02544/isvc-pmml-runtime-predictor-78c49cb5ff-h4r8f"
Oct 30 22:55:31.081224 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:55:31.081151    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-569vf\" (UniqueName: \"kubernetes.io/projected/8eedbf1b-b2f9-4ee0-a610-3dcfeec0d5e6-kube-api-access-569vf\") pod \"isvc-pmml-runtime-predictor-78c49cb5ff-h4r8f\" (UID: \"8eedbf1b-b2f9-4ee0-a610-3dcfeec0d5e6\") " pod="e2e-gw1-7476d02544/isvc-pmml-runtime-predictor-78c49cb5ff-h4r8f"
Oct 30 22:55:31.129653 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:55:31.129613    4412 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-7476d02544/isvc-pmml-runtime-predictor-78c49cb5ff-h4r8f"
Oct 30 22:55:31.559605 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:55:31.559579    4412 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw1-7476d02544/isvc-pmml-runtime-predictor-78c49cb5ff-h4r8f"]
Oct 30 22:55:31.561624 ip-10-0-131-202 kubenswrapper[4412]: W1030 22:55:31.561591    4412 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8eedbf1b_b2f9_4ee0_a610_3dcfeec0d5e6.slice/crio-43c9d31247f2ac606f0534d79f212975350b1e5aeadff8397ef351dc7a85e828 WatchSource:0}: Error finding container 43c9d31247f2ac606f0534d79f212975350b1e5aeadff8397ef351dc7a85e828: Status 404 returned error can't find the container with id 43c9d31247f2ac606f0534d79f212975350b1e5aeadff8397ef351dc7a85e828
Oct 30 22:55:31.568874 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:55:31.568851    4412 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-7476d02544/isvc-pmml-runtime-predictor-78c49cb5ff-h4r8f" event={"ID":"8eedbf1b-b2f9-4ee0-a610-3dcfeec0d5e6","Type":"ContainerStarted","Data":"43c9d31247f2ac606f0534d79f212975350b1e5aeadff8397ef351dc7a85e828"}
Oct 30 22:55:32.573887 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:55:32.573848    4412 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-7476d02544/isvc-pmml-runtime-predictor-78c49cb5ff-h4r8f" event={"ID":"8eedbf1b-b2f9-4ee0-a610-3dcfeec0d5e6","Type":"ContainerStarted","Data":"5032fbab1ad620891c8e834cf42739d708d2d503bba82d07a01082e49a192fd2"}
Oct 30 22:55:34.449215 ip-10-0-131-202 kubenswrapper[4412]: E1030 22:55:34.449175    4412 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bkhmz" containerName="extractor"
Oct 30 22:55:36.589826 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:55:36.589792    4412 generic.go:356] "Generic (PLEG): container finished" podID="8eedbf1b-b2f9-4ee0-a610-3dcfeec0d5e6" containerID="5032fbab1ad620891c8e834cf42739d708d2d503bba82d07a01082e49a192fd2" exitCode=0
Oct 30 22:55:36.590241 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:55:36.589846    4412 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-7476d02544/isvc-pmml-runtime-predictor-78c49cb5ff-h4r8f" event={"ID":"8eedbf1b-b2f9-4ee0-a610-3dcfeec0d5e6","Type":"ContainerDied","Data":"5032fbab1ad620891c8e834cf42739d708d2d503bba82d07a01082e49a192fd2"}
Oct 30 22:55:37.596463 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:55:37.596432    4412 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-7476d02544/isvc-pmml-runtime-predictor-78c49cb5ff-h4r8f" event={"ID":"8eedbf1b-b2f9-4ee0-a610-3dcfeec0d5e6","Type":"ContainerStarted","Data":"b16a72abb61018fb61234ed8f5c36c33632871c2d9da0ab499b1ec41969bdbe4"}
Oct 30 22:55:37.596918 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:55:37.596490    4412 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-7476d02544/isvc-pmml-runtime-predictor-78c49cb5ff-h4r8f" event={"ID":"8eedbf1b-b2f9-4ee0-a610-3dcfeec0d5e6","Type":"ContainerStarted","Data":"8931e2a627e7515352a570359464013be9bb2936bc9c9d02f5a030df0fa14cdf"}
Oct 30 22:55:37.596918 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:55:37.596686    4412 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw1-7476d02544/isvc-pmml-runtime-predictor-78c49cb5ff-h4r8f"
Oct 30 22:55:37.716686 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:55:37.716633    4412 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-gw1-7476d02544/isvc-pmml-runtime-predictor-78c49cb5ff-h4r8f" podStartSLOduration=3.6541535720000002 podStartE2EDuration="7.716619099s" totalImagesPullingTime="0s" totalInitContainerRuntime="4.062465527s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:55:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:55:37.715684691 +0000 UTC m=+3758.220356642" watchObservedRunningTime="2026-10-30 22:55:37.716619099 +0000 UTC m=+3758.221291124"
Oct 30 22:55:38.599800 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:55:38.599767    4412 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw1-7476d02544/isvc-pmml-runtime-predictor-78c49cb5ff-h4r8f"
Oct 30 22:55:38.601052 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:55:38.601027    4412 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-7476d02544/isvc-pmml-runtime-predictor-78c49cb5ff-h4r8f" podUID="8eedbf1b-b2f9-4ee0-a610-3dcfeec0d5e6" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.41:8080: connect: connection refused"
Oct 30 22:55:39.603235 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:55:39.603188    4412 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-7476d02544/isvc-pmml-runtime-predictor-78c49cb5ff-h4r8f" podUID="8eedbf1b-b2f9-4ee0-a610-3dcfeec0d5e6" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.41:8080: connect: connection refused"
Oct 30 22:55:42.548771 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:55:42.548740    4412 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw0-37d062d1dd/isvc-sklearn-mcp-predictor-79545c9b5b-nmkbp"
Oct 30 22:55:44.607869 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:55:44.607840    4412 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw1-7476d02544/isvc-pmml-runtime-predictor-78c49cb5ff-h4r8f"
Oct 30 22:55:44.608399 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:55:44.608366    4412 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-7476d02544/isvc-pmml-runtime-predictor-78c49cb5ff-h4r8f" podUID="8eedbf1b-b2f9-4ee0-a610-3dcfeec0d5e6" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.41:8080: connect: connection refused"
Oct 30 22:55:53.082456 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:55:53.082423    4412 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw0-37d062d1dd/isvc-sklearn-mcp-predictor-79545c9b5b-nmkbp"]
Oct 30 22:55:53.082871 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:55:53.082679    4412 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw0-37d062d1dd/isvc-sklearn-mcp-predictor-79545c9b5b-nmkbp" podUID="ca32745c-b63f-4e11-9881-2cbb64726053" containerName="kserve-container" containerID="cri-o://699469970ab06bd63b82cd7e8d6f6615f08ecf2ba57443d32e2c7276e34fbca6" gracePeriod=30
Oct 30 22:55:53.082871 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:55:53.082746    4412 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw0-37d062d1dd/isvc-sklearn-mcp-predictor-79545c9b5b-nmkbp" podUID="ca32745c-b63f-4e11-9881-2cbb64726053" containerName="kube-rbac-proxy" containerID="cri-o://e3890b8f0a9d49910f26ea7d8a97f533eada1c5e0ebe195ddcf3e0a02fbe4ccc" gracePeriod=30
Oct 30 22:55:53.082871 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:55:53.082746    4412 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw0-37d062d1dd/isvc-sklearn-mcp-predictor-79545c9b5b-nmkbp" podUID="ca32745c-b63f-4e11-9881-2cbb64726053" containerName="kserve-agent" containerID="cri-o://37525ef5950a616805dbbbe8c07dd9fa20952f07652ee7324c0dc4851871a0c1" gracePeriod=30
Oct 30 22:55:53.658571 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:55:53.658531    4412 generic.go:356] "Generic (PLEG): container finished" podID="ca32745c-b63f-4e11-9881-2cbb64726053" containerID="e3890b8f0a9d49910f26ea7d8a97f533eada1c5e0ebe195ddcf3e0a02fbe4ccc" exitCode=0
Oct 30 22:55:53.658759 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:55:53.658600    4412 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-37d062d1dd/isvc-sklearn-mcp-predictor-79545c9b5b-nmkbp" event={"ID":"ca32745c-b63f-4e11-9881-2cbb64726053","Type":"ContainerDied","Data":"e3890b8f0a9d49910f26ea7d8a97f533eada1c5e0ebe195ddcf3e0a02fbe4ccc"}
Oct 30 22:55:54.608881 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:55:54.608846    4412 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-7476d02544/isvc-pmml-runtime-predictor-78c49cb5ff-h4r8f" podUID="8eedbf1b-b2f9-4ee0-a610-3dcfeec0d5e6" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.41:8080: connect: connection refused"
Oct 30 22:55:55.669385 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:55:55.669304    4412 generic.go:356] "Generic (PLEG): container finished" podID="ca32745c-b63f-4e11-9881-2cbb64726053" containerID="699469970ab06bd63b82cd7e8d6f6615f08ecf2ba57443d32e2c7276e34fbca6" exitCode=0
Oct 30 22:55:55.669385 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:55:55.669372    4412 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-37d062d1dd/isvc-sklearn-mcp-predictor-79545c9b5b-nmkbp" event={"ID":"ca32745c-b63f-4e11-9881-2cbb64726053","Type":"ContainerDied","Data":"699469970ab06bd63b82cd7e8d6f6615f08ecf2ba57443d32e2c7276e34fbca6"}
Oct 30 22:55:57.354899 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:55:57.354862    4412 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-37d062d1dd/isvc-sklearn-mcp-predictor-79545c9b5b-nmkbp" podUID="ca32745c-b63f-4e11-9881-2cbb64726053" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.40:8643/healthz\": dial tcp 10.133.0.40:8643: connect: connection refused"
Oct 30 22:56:02.354674 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:56:02.354632    4412 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-37d062d1dd/isvc-sklearn-mcp-predictor-79545c9b5b-nmkbp" podUID="ca32745c-b63f-4e11-9881-2cbb64726053" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.40:8643/healthz\": dial tcp 10.133.0.40:8643: connect: connection refused"
Oct 30 22:56:02.361112 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:56:02.361084    4412 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-37d062d1dd/isvc-sklearn-mcp-predictor-79545c9b5b-nmkbp" podUID="ca32745c-b63f-4e11-9881-2cbb64726053" containerName="kserve-container" probeResult="failure" output="Get \"http://10.133.0.40:8080/v1/models/isvc-sklearn-mcp\": dial tcp 10.133.0.40:8080: connect: connection refused"
Oct 30 22:56:04.449390 ip-10-0-131-202 kubenswrapper[4412]: E1030 22:56:04.449355    4412 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bkhmz" containerName="extractor"
Oct 30 22:56:04.608762 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:56:04.608729    4412 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-7476d02544/isvc-pmml-runtime-predictor-78c49cb5ff-h4r8f" podUID="8eedbf1b-b2f9-4ee0-a610-3dcfeec0d5e6" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.41:8080: connect: connection refused"
Oct 30 22:56:07.355013 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:56:07.354971    4412 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-37d062d1dd/isvc-sklearn-mcp-predictor-79545c9b5b-nmkbp" podUID="ca32745c-b63f-4e11-9881-2cbb64726053" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.40:8643/healthz\": dial tcp 10.133.0.40:8643: connect: connection refused"
Oct 30 22:56:07.355423 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:56:07.355106    4412 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw0-37d062d1dd/isvc-sklearn-mcp-predictor-79545c9b5b-nmkbp"
Oct 30 22:56:12.355189 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:56:12.355144    4412 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-37d062d1dd/isvc-sklearn-mcp-predictor-79545c9b5b-nmkbp" podUID="ca32745c-b63f-4e11-9881-2cbb64726053" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.40:8643/healthz\": dial tcp 10.133.0.40:8643: connect: connection refused"
Oct 30 22:56:12.361569 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:56:12.361543    4412 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-37d062d1dd/isvc-sklearn-mcp-predictor-79545c9b5b-nmkbp" podUID="ca32745c-b63f-4e11-9881-2cbb64726053" containerName="kserve-container" probeResult="failure" output="Get \"http://10.133.0.40:8080/v1/models/isvc-sklearn-mcp\": dial tcp 10.133.0.40:8080: connect: connection refused"
Oct 30 22:56:14.608923 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:56:14.608884    4412 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-7476d02544/isvc-pmml-runtime-predictor-78c49cb5ff-h4r8f" podUID="8eedbf1b-b2f9-4ee0-a610-3dcfeec0d5e6" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.41:8080: connect: connection refused"
Oct 30 22:56:17.354962 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:56:17.354917    4412 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-37d062d1dd/isvc-sklearn-mcp-predictor-79545c9b5b-nmkbp" podUID="ca32745c-b63f-4e11-9881-2cbb64726053" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.40:8643/healthz\": dial tcp 10.133.0.40:8643: connect: connection refused"
Oct 30 22:56:22.355523 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:56:22.355483    4412 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-37d062d1dd/isvc-sklearn-mcp-predictor-79545c9b5b-nmkbp" podUID="ca32745c-b63f-4e11-9881-2cbb64726053" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.40:8643/healthz\": dial tcp 10.133.0.40:8643: connect: connection refused"
Oct 30 22:56:22.361082 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:56:22.361058    4412 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-37d062d1dd/isvc-sklearn-mcp-predictor-79545c9b5b-nmkbp" podUID="ca32745c-b63f-4e11-9881-2cbb64726053" containerName="kserve-container" probeResult="failure" output="Get \"http://10.133.0.40:8080/v1/models/isvc-sklearn-mcp\": dial tcp 10.133.0.40:8080: connect: connection refused"
Oct 30 22:56:22.361196 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:56:22.361185    4412 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw0-37d062d1dd/isvc-sklearn-mcp-predictor-79545c9b5b-nmkbp"
Oct 30 22:56:23.230697 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:56:23.230675    4412 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-37d062d1dd/isvc-sklearn-mcp-predictor-79545c9b5b-nmkbp"
Oct 30 22:56:23.334160 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:56:23.334108    4412 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ca32745c-b63f-4e11-9881-2cbb64726053-kube-api-access-rx62n\" (UniqueName: \"kubernetes.io/projected/ca32745c-b63f-4e11-9881-2cbb64726053-kube-api-access-rx62n\") pod \"ca32745c-b63f-4e11-9881-2cbb64726053\" (UID: \"ca32745c-b63f-4e11-9881-2cbb64726053\") "
Oct 30 22:56:23.334334 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:56:23.334191    4412 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ca32745c-b63f-4e11-9881-2cbb64726053-kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/ca32745c-b63f-4e11-9881-2cbb64726053-kserve-provision-location\") pod \"ca32745c-b63f-4e11-9881-2cbb64726053\" (UID: \"ca32745c-b63f-4e11-9881-2cbb64726053\") "
Oct 30 22:56:23.334334 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:56:23.334248    4412 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ca32745c-b63f-4e11-9881-2cbb64726053-kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/ca32745c-b63f-4e11-9881-2cbb64726053-kube-rbac-proxy-sar-config\") pod \"ca32745c-b63f-4e11-9881-2cbb64726053\" (UID: \"ca32745c-b63f-4e11-9881-2cbb64726053\") "
Oct 30 22:56:23.334334 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:56:23.334278    4412 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ca32745c-b63f-4e11-9881-2cbb64726053-cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/ca32745c-b63f-4e11-9881-2cbb64726053-cabundle-cert\") pod \"ca32745c-b63f-4e11-9881-2cbb64726053\" (UID: \"ca32745c-b63f-4e11-9881-2cbb64726053\") "
Oct 30 22:56:23.334334 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:56:23.334306    4412 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/ca32745c-b63f-4e11-9881-2cbb64726053-proxy-tls\" (UniqueName: \"kubernetes.io/secret/ca32745c-b63f-4e11-9881-2cbb64726053-proxy-tls\") pod \"ca32745c-b63f-4e11-9881-2cbb64726053\" (UID: \"ca32745c-b63f-4e11-9881-2cbb64726053\") "
Oct 30 22:56:23.334650 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:56:23.334618    4412 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ca32745c-b63f-4e11-9881-2cbb64726053-kserve-provision-location" pod "ca32745c-b63f-4e11-9881-2cbb64726053" (UID: "ca32745c-b63f-4e11-9881-2cbb64726053"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:56:23.334650 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:56:23.334642    4412 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ca32745c-b63f-4e11-9881-2cbb64726053-kube-rbac-proxy-sar-config" pod "ca32745c-b63f-4e11-9881-2cbb64726053" (UID: "ca32745c-b63f-4e11-9881-2cbb64726053"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:56:23.334787 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:56:23.334683    4412 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ca32745c-b63f-4e11-9881-2cbb64726053-cabundle-cert" pod "ca32745c-b63f-4e11-9881-2cbb64726053" (UID: "ca32745c-b63f-4e11-9881-2cbb64726053"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:56:23.336493 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:56:23.336468    4412 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ca32745c-b63f-4e11-9881-2cbb64726053-kube-api-access-rx62n" pod "ca32745c-b63f-4e11-9881-2cbb64726053" (UID: "ca32745c-b63f-4e11-9881-2cbb64726053"). InnerVolumeSpecName "kube-api-access-rx62n". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:56:23.336493 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:56:23.336482    4412 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ca32745c-b63f-4e11-9881-2cbb64726053-proxy-tls" pod "ca32745c-b63f-4e11-9881-2cbb64726053" (UID: "ca32745c-b63f-4e11-9881-2cbb64726053"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:56:23.435503 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:56:23.435475    4412 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/ca32745c-b63f-4e11-9881-2cbb64726053-kserve-provision-location\") on node \"ip-10-0-131-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:56:23.435503 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:56:23.435501    4412 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/ca32745c-b63f-4e11-9881-2cbb64726053-kube-rbac-proxy-sar-config\") on node \"ip-10-0-131-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:56:23.435503 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:56:23.435511    4412 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/ca32745c-b63f-4e11-9881-2cbb64726053-cabundle-cert\") on node \"ip-10-0-131-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:56:23.435957 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:56:23.435522    4412 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/ca32745c-b63f-4e11-9881-2cbb64726053-proxy-tls\") on node \"ip-10-0-131-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:56:23.435957 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:56:23.435531    4412 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rx62n\" (UniqueName: \"kubernetes.io/projected/ca32745c-b63f-4e11-9881-2cbb64726053-kube-api-access-rx62n\") on node \"ip-10-0-131-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:56:23.773159 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:56:23.773108    4412 generic.go:356] "Generic (PLEG): container finished" podID="ca32745c-b63f-4e11-9881-2cbb64726053" containerID="37525ef5950a616805dbbbe8c07dd9fa20952f07652ee7324c0dc4851871a0c1" exitCode=0
Oct 30 22:56:23.773159 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:56:23.773155    4412 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-37d062d1dd/isvc-sklearn-mcp-predictor-79545c9b5b-nmkbp" event={"ID":"ca32745c-b63f-4e11-9881-2cbb64726053","Type":"ContainerDied","Data":"37525ef5950a616805dbbbe8c07dd9fa20952f07652ee7324c0dc4851871a0c1"}
Oct 30 22:56:23.773410 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:56:23.773193    4412 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-37d062d1dd/isvc-sklearn-mcp-predictor-79545c9b5b-nmkbp"
Oct 30 22:56:23.773410 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:56:23.773222    4412 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-37d062d1dd/isvc-sklearn-mcp-predictor-79545c9b5b-nmkbp" event={"ID":"ca32745c-b63f-4e11-9881-2cbb64726053","Type":"ContainerDied","Data":"b3da522c7dd205eac60c7e306f60ca851abddaa2e53b450f6f5f1d4a81ddb9a7"}
Oct 30 22:56:23.773410 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:56:23.773253    4412 scope.go:122] "RemoveContainer" containerID="e3890b8f0a9d49910f26ea7d8a97f533eada1c5e0ebe195ddcf3e0a02fbe4ccc"
Oct 30 22:56:23.781344 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:56:23.781329    4412 scope.go:122] "RemoveContainer" containerID="37525ef5950a616805dbbbe8c07dd9fa20952f07652ee7324c0dc4851871a0c1"
Oct 30 22:56:23.788679 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:56:23.788664    4412 scope.go:122] "RemoveContainer" containerID="699469970ab06bd63b82cd7e8d6f6615f08ecf2ba57443d32e2c7276e34fbca6"
Oct 30 22:56:23.795798 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:56:23.795781    4412 scope.go:122] "RemoveContainer" containerID="f381b7850ab6052d3b9c16e572191d3da1a7ca3a288333a3b97f1347cc3edc3b"
Oct 30 22:56:23.802904 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:56:23.802887    4412 scope.go:122] "RemoveContainer" containerID="e3890b8f0a9d49910f26ea7d8a97f533eada1c5e0ebe195ddcf3e0a02fbe4ccc"
Oct 30 22:56:23.803173 ip-10-0-131-202 kubenswrapper[4412]: E1030 22:56:23.803150    4412 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e3890b8f0a9d49910f26ea7d8a97f533eada1c5e0ebe195ddcf3e0a02fbe4ccc\": container with ID starting with e3890b8f0a9d49910f26ea7d8a97f533eada1c5e0ebe195ddcf3e0a02fbe4ccc not found: ID does not exist" containerID="e3890b8f0a9d49910f26ea7d8a97f533eada1c5e0ebe195ddcf3e0a02fbe4ccc"
Oct 30 22:56:23.803260 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:56:23.803181    4412 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e3890b8f0a9d49910f26ea7d8a97f533eada1c5e0ebe195ddcf3e0a02fbe4ccc"} err="failed to get container status \"e3890b8f0a9d49910f26ea7d8a97f533eada1c5e0ebe195ddcf3e0a02fbe4ccc\": rpc error: code = NotFound desc = could not find container \"e3890b8f0a9d49910f26ea7d8a97f533eada1c5e0ebe195ddcf3e0a02fbe4ccc\": container with ID starting with e3890b8f0a9d49910f26ea7d8a97f533eada1c5e0ebe195ddcf3e0a02fbe4ccc not found: ID does not exist"
Oct 30 22:56:23.803260 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:56:23.803198    4412 scope.go:122] "RemoveContainer" containerID="37525ef5950a616805dbbbe8c07dd9fa20952f07652ee7324c0dc4851871a0c1"
Oct 30 22:56:23.803451 ip-10-0-131-202 kubenswrapper[4412]: E1030 22:56:23.803436    4412 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"37525ef5950a616805dbbbe8c07dd9fa20952f07652ee7324c0dc4851871a0c1\": container with ID starting with 37525ef5950a616805dbbbe8c07dd9fa20952f07652ee7324c0dc4851871a0c1 not found: ID does not exist" containerID="37525ef5950a616805dbbbe8c07dd9fa20952f07652ee7324c0dc4851871a0c1"
Oct 30 22:56:23.803491 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:56:23.803455    4412 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"37525ef5950a616805dbbbe8c07dd9fa20952f07652ee7324c0dc4851871a0c1"} err="failed to get container status \"37525ef5950a616805dbbbe8c07dd9fa20952f07652ee7324c0dc4851871a0c1\": rpc error: code = NotFound desc = could not find container \"37525ef5950a616805dbbbe8c07dd9fa20952f07652ee7324c0dc4851871a0c1\": container with ID starting with 37525ef5950a616805dbbbe8c07dd9fa20952f07652ee7324c0dc4851871a0c1 not found: ID does not exist"
Oct 30 22:56:23.803491 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:56:23.803467    4412 scope.go:122] "RemoveContainer" containerID="699469970ab06bd63b82cd7e8d6f6615f08ecf2ba57443d32e2c7276e34fbca6"
Oct 30 22:56:23.803667 ip-10-0-131-202 kubenswrapper[4412]: E1030 22:56:23.803652    4412 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"699469970ab06bd63b82cd7e8d6f6615f08ecf2ba57443d32e2c7276e34fbca6\": container with ID starting with 699469970ab06bd63b82cd7e8d6f6615f08ecf2ba57443d32e2c7276e34fbca6 not found: ID does not exist" containerID="699469970ab06bd63b82cd7e8d6f6615f08ecf2ba57443d32e2c7276e34fbca6"
Oct 30 22:56:23.803708 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:56:23.803671    4412 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"699469970ab06bd63b82cd7e8d6f6615f08ecf2ba57443d32e2c7276e34fbca6"} err="failed to get container status \"699469970ab06bd63b82cd7e8d6f6615f08ecf2ba57443d32e2c7276e34fbca6\": rpc error: code = NotFound desc = could not find container \"699469970ab06bd63b82cd7e8d6f6615f08ecf2ba57443d32e2c7276e34fbca6\": container with ID starting with 699469970ab06bd63b82cd7e8d6f6615f08ecf2ba57443d32e2c7276e34fbca6 not found: ID does not exist"
Oct 30 22:56:23.803708 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:56:23.803684    4412 scope.go:122] "RemoveContainer" containerID="f381b7850ab6052d3b9c16e572191d3da1a7ca3a288333a3b97f1347cc3edc3b"
Oct 30 22:56:23.803864 ip-10-0-131-202 kubenswrapper[4412]: E1030 22:56:23.803850    4412 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f381b7850ab6052d3b9c16e572191d3da1a7ca3a288333a3b97f1347cc3edc3b\": container with ID starting with f381b7850ab6052d3b9c16e572191d3da1a7ca3a288333a3b97f1347cc3edc3b not found: ID does not exist" containerID="f381b7850ab6052d3b9c16e572191d3da1a7ca3a288333a3b97f1347cc3edc3b"
Oct 30 22:56:23.803902 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:56:23.803866    4412 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f381b7850ab6052d3b9c16e572191d3da1a7ca3a288333a3b97f1347cc3edc3b"} err="failed to get container status \"f381b7850ab6052d3b9c16e572191d3da1a7ca3a288333a3b97f1347cc3edc3b\": rpc error: code = NotFound desc = could not find container \"f381b7850ab6052d3b9c16e572191d3da1a7ca3a288333a3b97f1347cc3edc3b\": container with ID starting with f381b7850ab6052d3b9c16e572191d3da1a7ca3a288333a3b97f1347cc3edc3b not found: ID does not exist"
Oct 30 22:56:23.939701 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:56:23.939626    4412 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw0-37d062d1dd/isvc-sklearn-mcp-predictor-79545c9b5b-nmkbp"]
Oct 30 22:56:23.990267 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:56:23.990237    4412 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-gw0-37d062d1dd/isvc-sklearn-mcp-predictor-79545c9b5b-nmkbp"]
Oct 30 22:56:24.608524 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:56:24.608482    4412 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-7476d02544/isvc-pmml-runtime-predictor-78c49cb5ff-h4r8f" podUID="8eedbf1b-b2f9-4ee0-a610-3dcfeec0d5e6" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.41:8080: connect: connection refused"
Oct 30 22:56:25.625678 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:56:25.625649    4412 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ca32745c-b63f-4e11-9881-2cbb64726053" path="/var/lib/kubelet/pods/ca32745c-b63f-4e11-9881-2cbb64726053/volumes"
Oct 30 22:56:34.450539 ip-10-0-131-202 kubenswrapper[4412]: E1030 22:56:34.450499    4412 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bkhmz" containerName="extractor"
Oct 30 22:56:34.608883 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:56:34.608846    4412 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-7476d02544/isvc-pmml-runtime-predictor-78c49cb5ff-h4r8f" podUID="8eedbf1b-b2f9-4ee0-a610-3dcfeec0d5e6" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.41:8080: connect: connection refused"
Oct 30 22:56:44.609138 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:56:44.609036    4412 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-7476d02544/isvc-pmml-runtime-predictor-78c49cb5ff-h4r8f" podUID="8eedbf1b-b2f9-4ee0-a610-3dcfeec0d5e6" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.41:8080: connect: connection refused"
Oct 30 22:56:54.609276 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:56:54.609235    4412 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-7476d02544/isvc-pmml-runtime-predictor-78c49cb5ff-h4r8f" podUID="8eedbf1b-b2f9-4ee0-a610-3dcfeec0d5e6" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.41:8080: connect: connection refused"
Oct 30 22:56:55.622296 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:56:55.622248    4412 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-7476d02544/isvc-pmml-runtime-predictor-78c49cb5ff-h4r8f" podUID="8eedbf1b-b2f9-4ee0-a610-3dcfeec0d5e6" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.41:8080: connect: connection refused"
Oct 30 22:57:04.450929 ip-10-0-131-202 kubenswrapper[4412]: E1030 22:57:04.450885    4412 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bkhmz" containerName="extractor"
Oct 30 22:57:05.626048 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:57:05.626024    4412 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw1-7476d02544/isvc-pmml-runtime-predictor-78c49cb5ff-h4r8f"
Oct 30 22:57:12.790651 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:57:12.790618    4412 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw1-7476d02544/isvc-pmml-runtime-predictor-78c49cb5ff-h4r8f"]
Oct 30 22:57:12.791115 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:57:12.790915    4412 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw1-7476d02544/isvc-pmml-runtime-predictor-78c49cb5ff-h4r8f" podUID="8eedbf1b-b2f9-4ee0-a610-3dcfeec0d5e6" containerName="kserve-container" containerID="cri-o://8931e2a627e7515352a570359464013be9bb2936bc9c9d02f5a030df0fa14cdf" gracePeriod=30
Oct 30 22:57:12.791115 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:57:12.790950    4412 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw1-7476d02544/isvc-pmml-runtime-predictor-78c49cb5ff-h4r8f" podUID="8eedbf1b-b2f9-4ee0-a610-3dcfeec0d5e6" containerName="kube-rbac-proxy" containerID="cri-o://b16a72abb61018fb61234ed8f5c36c33632871c2d9da0ab499b1ec41969bdbe4" gracePeriod=30
Oct 30 22:57:12.958188 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:57:12.958158    4412 generic.go:356] "Generic (PLEG): container finished" podID="8eedbf1b-b2f9-4ee0-a610-3dcfeec0d5e6" containerID="b16a72abb61018fb61234ed8f5c36c33632871c2d9da0ab499b1ec41969bdbe4" exitCode=0
Oct 30 22:57:12.958333 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:57:12.958210    4412 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-7476d02544/isvc-pmml-runtime-predictor-78c49cb5ff-h4r8f" event={"ID":"8eedbf1b-b2f9-4ee0-a610-3dcfeec0d5e6","Type":"ContainerDied","Data":"b16a72abb61018fb61234ed8f5c36c33632871c2d9da0ab499b1ec41969bdbe4"}
Oct 30 22:57:14.604179 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:57:14.604141    4412 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-7476d02544/isvc-pmml-runtime-predictor-78c49cb5ff-h4r8f" podUID="8eedbf1b-b2f9-4ee0-a610-3dcfeec0d5e6" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.41:8643/healthz\": dial tcp 10.133.0.41:8643: connect: connection refused"
Oct 30 22:57:15.623010 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:57:15.622972    4412 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-7476d02544/isvc-pmml-runtime-predictor-78c49cb5ff-h4r8f" podUID="8eedbf1b-b2f9-4ee0-a610-3dcfeec0d5e6" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.41:8080: connect: connection refused"
Oct 30 22:57:16.729517 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:57:16.729491    4412 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-7476d02544/isvc-pmml-runtime-predictor-78c49cb5ff-h4r8f"
Oct 30 22:57:16.773733 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:57:16.773709    4412 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8eedbf1b-b2f9-4ee0-a610-3dcfeec0d5e6-kube-api-access-569vf\" (UniqueName: \"kubernetes.io/projected/8eedbf1b-b2f9-4ee0-a610-3dcfeec0d5e6-kube-api-access-569vf\") pod \"8eedbf1b-b2f9-4ee0-a610-3dcfeec0d5e6\" (UID: \"8eedbf1b-b2f9-4ee0-a610-3dcfeec0d5e6\") "
Oct 30 22:57:16.773844 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:57:16.773754    4412 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/8eedbf1b-b2f9-4ee0-a610-3dcfeec0d5e6-cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/8eedbf1b-b2f9-4ee0-a610-3dcfeec0d5e6-cabundle-cert\") pod \"8eedbf1b-b2f9-4ee0-a610-3dcfeec0d5e6\" (UID: \"8eedbf1b-b2f9-4ee0-a610-3dcfeec0d5e6\") "
Oct 30 22:57:16.773844 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:57:16.773835    4412 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/8eedbf1b-b2f9-4ee0-a610-3dcfeec0d5e6-kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/8eedbf1b-b2f9-4ee0-a610-3dcfeec0d5e6-kube-rbac-proxy-sar-config\") pod \"8eedbf1b-b2f9-4ee0-a610-3dcfeec0d5e6\" (UID: \"8eedbf1b-b2f9-4ee0-a610-3dcfeec0d5e6\") "
Oct 30 22:57:16.773955 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:57:16.773864    4412 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8eedbf1b-b2f9-4ee0-a610-3dcfeec0d5e6-kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/8eedbf1b-b2f9-4ee0-a610-3dcfeec0d5e6-kserve-provision-location\") pod \"8eedbf1b-b2f9-4ee0-a610-3dcfeec0d5e6\" (UID: \"8eedbf1b-b2f9-4ee0-a610-3dcfeec0d5e6\") "
Oct 30 22:57:16.773955 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:57:16.773913    4412 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/8eedbf1b-b2f9-4ee0-a610-3dcfeec0d5e6-proxy-tls\" (UniqueName: \"kubernetes.io/secret/8eedbf1b-b2f9-4ee0-a610-3dcfeec0d5e6-proxy-tls\") pod \"8eedbf1b-b2f9-4ee0-a610-3dcfeec0d5e6\" (UID: \"8eedbf1b-b2f9-4ee0-a610-3dcfeec0d5e6\") "
Oct 30 22:57:16.774279 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:57:16.774238    4412 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8eedbf1b-b2f9-4ee0-a610-3dcfeec0d5e6-cabundle-cert" pod "8eedbf1b-b2f9-4ee0-a610-3dcfeec0d5e6" (UID: "8eedbf1b-b2f9-4ee0-a610-3dcfeec0d5e6"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:57:16.774279 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:57:16.774252    4412 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8eedbf1b-b2f9-4ee0-a610-3dcfeec0d5e6-kserve-provision-location" pod "8eedbf1b-b2f9-4ee0-a610-3dcfeec0d5e6" (UID: "8eedbf1b-b2f9-4ee0-a610-3dcfeec0d5e6"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:57:16.774279 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:57:16.774257    4412 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8eedbf1b-b2f9-4ee0-a610-3dcfeec0d5e6-kube-rbac-proxy-sar-config" pod "8eedbf1b-b2f9-4ee0-a610-3dcfeec0d5e6" (UID: "8eedbf1b-b2f9-4ee0-a610-3dcfeec0d5e6"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:57:16.775871 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:57:16.775850    4412 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8eedbf1b-b2f9-4ee0-a610-3dcfeec0d5e6-kube-api-access-569vf" pod "8eedbf1b-b2f9-4ee0-a610-3dcfeec0d5e6" (UID: "8eedbf1b-b2f9-4ee0-a610-3dcfeec0d5e6"). InnerVolumeSpecName "kube-api-access-569vf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:57:16.776004 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:57:16.775977    4412 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8eedbf1b-b2f9-4ee0-a610-3dcfeec0d5e6-proxy-tls" pod "8eedbf1b-b2f9-4ee0-a610-3dcfeec0d5e6" (UID: "8eedbf1b-b2f9-4ee0-a610-3dcfeec0d5e6"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:57:16.874808 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:57:16.874749    4412 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/8eedbf1b-b2f9-4ee0-a610-3dcfeec0d5e6-kube-rbac-proxy-sar-config\") on node \"ip-10-0-131-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:57:16.874808 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:57:16.874771    4412 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/8eedbf1b-b2f9-4ee0-a610-3dcfeec0d5e6-kserve-provision-location\") on node \"ip-10-0-131-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:57:16.874808 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:57:16.874784    4412 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/8eedbf1b-b2f9-4ee0-a610-3dcfeec0d5e6-proxy-tls\") on node \"ip-10-0-131-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:57:16.874808 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:57:16.874797    4412 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-569vf\" (UniqueName: \"kubernetes.io/projected/8eedbf1b-b2f9-4ee0-a610-3dcfeec0d5e6-kube-api-access-569vf\") on node \"ip-10-0-131-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:57:16.874808 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:57:16.874806    4412 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/8eedbf1b-b2f9-4ee0-a610-3dcfeec0d5e6-cabundle-cert\") on node \"ip-10-0-131-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:57:16.976016 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:57:16.975975    4412 generic.go:356] "Generic (PLEG): container finished" podID="8eedbf1b-b2f9-4ee0-a610-3dcfeec0d5e6" containerID="8931e2a627e7515352a570359464013be9bb2936bc9c9d02f5a030df0fa14cdf" exitCode=0
Oct 30 22:57:16.976179 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:57:16.976024    4412 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-7476d02544/isvc-pmml-runtime-predictor-78c49cb5ff-h4r8f" event={"ID":"8eedbf1b-b2f9-4ee0-a610-3dcfeec0d5e6","Type":"ContainerDied","Data":"8931e2a627e7515352a570359464013be9bb2936bc9c9d02f5a030df0fa14cdf"}
Oct 30 22:57:16.976179 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:57:16.976051    4412 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-7476d02544/isvc-pmml-runtime-predictor-78c49cb5ff-h4r8f"
Oct 30 22:57:16.976179 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:57:16.976134    4412 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-7476d02544/isvc-pmml-runtime-predictor-78c49cb5ff-h4r8f" event={"ID":"8eedbf1b-b2f9-4ee0-a610-3dcfeec0d5e6","Type":"ContainerDied","Data":"43c9d31247f2ac606f0534d79f212975350b1e5aeadff8397ef351dc7a85e828"}
Oct 30 22:57:16.976179 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:57:16.976174    4412 scope.go:122] "RemoveContainer" containerID="b16a72abb61018fb61234ed8f5c36c33632871c2d9da0ab499b1ec41969bdbe4"
Oct 30 22:57:16.984859 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:57:16.984839    4412 scope.go:122] "RemoveContainer" containerID="8931e2a627e7515352a570359464013be9bb2936bc9c9d02f5a030df0fa14cdf"
Oct 30 22:57:16.992256 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:57:16.992234    4412 scope.go:122] "RemoveContainer" containerID="5032fbab1ad620891c8e834cf42739d708d2d503bba82d07a01082e49a192fd2"
Oct 30 22:57:16.999446 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:57:16.999428    4412 scope.go:122] "RemoveContainer" containerID="b16a72abb61018fb61234ed8f5c36c33632871c2d9da0ab499b1ec41969bdbe4"
Oct 30 22:57:16.999693 ip-10-0-131-202 kubenswrapper[4412]: E1030 22:57:16.999673    4412 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b16a72abb61018fb61234ed8f5c36c33632871c2d9da0ab499b1ec41969bdbe4\": container with ID starting with b16a72abb61018fb61234ed8f5c36c33632871c2d9da0ab499b1ec41969bdbe4 not found: ID does not exist" containerID="b16a72abb61018fb61234ed8f5c36c33632871c2d9da0ab499b1ec41969bdbe4"
Oct 30 22:57:16.999769 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:57:16.999704    4412 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b16a72abb61018fb61234ed8f5c36c33632871c2d9da0ab499b1ec41969bdbe4"} err="failed to get container status \"b16a72abb61018fb61234ed8f5c36c33632871c2d9da0ab499b1ec41969bdbe4\": rpc error: code = NotFound desc = could not find container \"b16a72abb61018fb61234ed8f5c36c33632871c2d9da0ab499b1ec41969bdbe4\": container with ID starting with b16a72abb61018fb61234ed8f5c36c33632871c2d9da0ab499b1ec41969bdbe4 not found: ID does not exist"
Oct 30 22:57:16.999769 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:57:16.999728    4412 scope.go:122] "RemoveContainer" containerID="8931e2a627e7515352a570359464013be9bb2936bc9c9d02f5a030df0fa14cdf"
Oct 30 22:57:17.000004 ip-10-0-131-202 kubenswrapper[4412]: E1030 22:57:16.999978    4412 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8931e2a627e7515352a570359464013be9bb2936bc9c9d02f5a030df0fa14cdf\": container with ID starting with 8931e2a627e7515352a570359464013be9bb2936bc9c9d02f5a030df0fa14cdf not found: ID does not exist" containerID="8931e2a627e7515352a570359464013be9bb2936bc9c9d02f5a030df0fa14cdf"
Oct 30 22:57:17.000044 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:57:17.000012    4412 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8931e2a627e7515352a570359464013be9bb2936bc9c9d02f5a030df0fa14cdf"} err="failed to get container status \"8931e2a627e7515352a570359464013be9bb2936bc9c9d02f5a030df0fa14cdf\": rpc error: code = NotFound desc = could not find container \"8931e2a627e7515352a570359464013be9bb2936bc9c9d02f5a030df0fa14cdf\": container with ID starting with 8931e2a627e7515352a570359464013be9bb2936bc9c9d02f5a030df0fa14cdf not found: ID does not exist"
Oct 30 22:57:17.000044 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:57:17.000028    4412 scope.go:122] "RemoveContainer" containerID="5032fbab1ad620891c8e834cf42739d708d2d503bba82d07a01082e49a192fd2"
Oct 30 22:57:17.000280 ip-10-0-131-202 kubenswrapper[4412]: E1030 22:57:17.000263    4412 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5032fbab1ad620891c8e834cf42739d708d2d503bba82d07a01082e49a192fd2\": container with ID starting with 5032fbab1ad620891c8e834cf42739d708d2d503bba82d07a01082e49a192fd2 not found: ID does not exist" containerID="5032fbab1ad620891c8e834cf42739d708d2d503bba82d07a01082e49a192fd2"
Oct 30 22:57:17.000335 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:57:17.000286    4412 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5032fbab1ad620891c8e834cf42739d708d2d503bba82d07a01082e49a192fd2"} err="failed to get container status \"5032fbab1ad620891c8e834cf42739d708d2d503bba82d07a01082e49a192fd2\": rpc error: code = NotFound desc = could not find container \"5032fbab1ad620891c8e834cf42739d708d2d503bba82d07a01082e49a192fd2\": container with ID starting with 5032fbab1ad620891c8e834cf42739d708d2d503bba82d07a01082e49a192fd2 not found: ID does not exist"
Oct 30 22:57:17.143560 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:57:17.143474    4412 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw1-7476d02544/isvc-pmml-runtime-predictor-78c49cb5ff-h4r8f"]
Oct 30 22:57:17.148065 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:57:17.148040    4412 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-gw1-7476d02544/isvc-pmml-runtime-predictor-78c49cb5ff-h4r8f"]
Oct 30 22:57:17.625843 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:57:17.625812    4412 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="8eedbf1b-b2f9-4ee0-a610-3dcfeec0d5e6" path="/var/lib/kubelet/pods/8eedbf1b-b2f9-4ee0-a610-3dcfeec0d5e6/volumes"
Oct 30 22:57:19.667393 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:57:19.667363    4412 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-must-gather-4qnp4/must-gather-4xjh4"]
Oct 30 22:57:19.667869 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:57:19.667769    4412 cpu_manager.go:414] "containerMap: removing container" podUID="8eedbf1b-b2f9-4ee0-a610-3dcfeec0d5e6" containerName="kube-rbac-proxy"
Oct 30 22:57:19.667869 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:57:19.667786    4412 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8eedbf1b-b2f9-4ee0-a610-3dcfeec0d5e6" containerName="kube-rbac-proxy"
Oct 30 22:57:19.667869 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:57:19.667802    4412 cpu_manager.go:414] "containerMap: removing container" podUID="ca32745c-b63f-4e11-9881-2cbb64726053" containerName="kube-rbac-proxy"
Oct 30 22:57:19.667869 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:57:19.667810    4412 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ca32745c-b63f-4e11-9881-2cbb64726053" containerName="kube-rbac-proxy"
Oct 30 22:57:19.667869 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:57:19.667819    4412 cpu_manager.go:414] "containerMap: removing container" podUID="8eedbf1b-b2f9-4ee0-a610-3dcfeec0d5e6" containerName="storage-initializer"
Oct 30 22:57:19.667869 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:57:19.667826    4412 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8eedbf1b-b2f9-4ee0-a610-3dcfeec0d5e6" containerName="storage-initializer"
Oct 30 22:57:19.667869 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:57:19.667837    4412 cpu_manager.go:414] "containerMap: removing container" podUID="ca32745c-b63f-4e11-9881-2cbb64726053" containerName="kserve-container"
Oct 30 22:57:19.667869 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:57:19.667843    4412 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ca32745c-b63f-4e11-9881-2cbb64726053" containerName="kserve-container"
Oct 30 22:57:19.667869 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:57:19.667853    4412 cpu_manager.go:414] "containerMap: removing container" podUID="8eedbf1b-b2f9-4ee0-a610-3dcfeec0d5e6" containerName="kserve-container"
Oct 30 22:57:19.667869 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:57:19.667860    4412 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8eedbf1b-b2f9-4ee0-a610-3dcfeec0d5e6" containerName="kserve-container"
Oct 30 22:57:19.667869 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:57:19.667874    4412 cpu_manager.go:414] "containerMap: removing container" podUID="ca32745c-b63f-4e11-9881-2cbb64726053" containerName="kserve-agent"
Oct 30 22:57:19.668438 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:57:19.667881    4412 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ca32745c-b63f-4e11-9881-2cbb64726053" containerName="kserve-agent"
Oct 30 22:57:19.668438 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:57:19.667894    4412 cpu_manager.go:414] "containerMap: removing container" podUID="ca32745c-b63f-4e11-9881-2cbb64726053" containerName="storage-initializer"
Oct 30 22:57:19.668438 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:57:19.667901    4412 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ca32745c-b63f-4e11-9881-2cbb64726053" containerName="storage-initializer"
Oct 30 22:57:19.668438 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:57:19.667978    4412 memory_manager.go:365] "RemoveStaleState removing state" podUID="ca32745c-b63f-4e11-9881-2cbb64726053" containerName="kserve-container"
Oct 30 22:57:19.668438 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:57:19.667992    4412 memory_manager.go:365] "RemoveStaleState removing state" podUID="8eedbf1b-b2f9-4ee0-a610-3dcfeec0d5e6" containerName="kserve-container"
Oct 30 22:57:19.668438 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:57:19.668010    4412 memory_manager.go:365] "RemoveStaleState removing state" podUID="ca32745c-b63f-4e11-9881-2cbb64726053" containerName="kserve-agent"
Oct 30 22:57:19.668438 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:57:19.668025    4412 memory_manager.go:365] "RemoveStaleState removing state" podUID="8eedbf1b-b2f9-4ee0-a610-3dcfeec0d5e6" containerName="kube-rbac-proxy"
Oct 30 22:57:19.668438 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:57:19.668035    4412 memory_manager.go:365] "RemoveStaleState removing state" podUID="ca32745c-b63f-4e11-9881-2cbb64726053" containerName="kube-rbac-proxy"
Oct 30 22:57:19.672608 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:57:19.672585    4412 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-4qnp4/must-gather-4xjh4"
Oct 30 22:57:19.726776 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:57:19.726734    4412 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-4qnp4\"/\"openshift-service-ca.crt\""
Oct 30 22:57:19.727633 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:57:19.727609    4412 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-4qnp4/must-gather-4xjh4"]
Oct 30 22:57:19.777035 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:57:19.777009    4412 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-4qnp4\"/\"kube-root-ca.crt\""
Oct 30 22:57:19.777179 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:57:19.777098    4412 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-4qnp4\"/\"default-dockercfg-s28gr\""
Oct 30 22:57:19.795377 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:57:19.795352    4412 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/f4417fed-9bde-4318-a027-f77843fce93b-must-gather-output\") pod \"must-gather-4xjh4\" (UID: \"f4417fed-9bde-4318-a027-f77843fce93b\") " pod="openshift-must-gather-4qnp4/must-gather-4xjh4"
Oct 30 22:57:19.795468 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:57:19.795386    4412 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gm7h6\" (UniqueName: \"kubernetes.io/projected/f4417fed-9bde-4318-a027-f77843fce93b-kube-api-access-gm7h6\") pod \"must-gather-4xjh4\" (UID: \"f4417fed-9bde-4318-a027-f77843fce93b\") " pod="openshift-must-gather-4qnp4/must-gather-4xjh4"
Oct 30 22:57:19.895868 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:57:19.895844    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/f4417fed-9bde-4318-a027-f77843fce93b-must-gather-output\") pod \"must-gather-4xjh4\" (UID: \"f4417fed-9bde-4318-a027-f77843fce93b\") " pod="openshift-must-gather-4qnp4/must-gather-4xjh4"
Oct 30 22:57:19.896018 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:57:19.895872    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gm7h6\" (UniqueName: \"kubernetes.io/projected/f4417fed-9bde-4318-a027-f77843fce93b-kube-api-access-gm7h6\") pod \"must-gather-4xjh4\" (UID: \"f4417fed-9bde-4318-a027-f77843fce93b\") " pod="openshift-must-gather-4qnp4/must-gather-4xjh4"
Oct 30 22:57:19.896236 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:57:19.896212    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/f4417fed-9bde-4318-a027-f77843fce93b-must-gather-output\") pod \"must-gather-4xjh4\" (UID: \"f4417fed-9bde-4318-a027-f77843fce93b\") " pod="openshift-must-gather-4qnp4/must-gather-4xjh4"
Oct 30 22:57:19.954186 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:57:19.954097    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gm7h6\" (UniqueName: \"kubernetes.io/projected/f4417fed-9bde-4318-a027-f77843fce93b-kube-api-access-gm7h6\") pod \"must-gather-4xjh4\" (UID: \"f4417fed-9bde-4318-a027-f77843fce93b\") " pod="openshift-must-gather-4qnp4/must-gather-4xjh4"
Oct 30 22:57:19.995242 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:57:19.995219    4412 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-4qnp4/must-gather-4xjh4"
Oct 30 22:57:20.365478 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:57:20.365422    4412 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-4qnp4/must-gather-4xjh4"]
Oct 30 22:57:20.368332 ip-10-0-131-202 kubenswrapper[4412]: W1030 22:57:20.368308    4412 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf4417fed_9bde_4318_a027_f77843fce93b.slice/crio-4aad5827ac26cc7a07647d7b0bf43f515938d78e6215a29f4417a4a743b2a4a0 WatchSource:0}: Error finding container 4aad5827ac26cc7a07647d7b0bf43f515938d78e6215a29f4417a4a743b2a4a0: Status 404 returned error can't find the container with id 4aad5827ac26cc7a07647d7b0bf43f515938d78e6215a29f4417a4a743b2a4a0
Oct 30 22:57:20.370528 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:57:20.370514    4412 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 22:57:20.991692 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:57:20.991659    4412 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-4qnp4/must-gather-4xjh4" event={"ID":"f4417fed-9bde-4318-a027-f77843fce93b","Type":"ContainerStarted","Data":"4aad5827ac26cc7a07647d7b0bf43f515938d78e6215a29f4417a4a743b2a4a0"}
Oct 30 22:57:28.042778 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:57:28.042743    4412 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-4qnp4/must-gather-4xjh4" event={"ID":"f4417fed-9bde-4318-a027-f77843fce93b","Type":"ContainerStarted","Data":"94fa88a98a55e90b474871b925d71c307ea4c2da7d55564a2a14420579fa294d"}
Oct 30 22:57:28.043205 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:57:28.042790    4412 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-4qnp4/must-gather-4xjh4" event={"ID":"f4417fed-9bde-4318-a027-f77843fce93b","Type":"ContainerStarted","Data":"e20c295c68050d4928df195cbcf1d1ee9b373192519f5190217b2347215e2c45"}
Oct 30 22:57:28.160061 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:57:28.160004    4412 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-must-gather-4qnp4/must-gather-4xjh4" podStartSLOduration=2.036051131 podStartE2EDuration="9.159982615s" totalImagesPullingTime="7.123931484s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:57:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:57:28.158595711 +0000 UTC m=+3868.663267663" watchObservedRunningTime="2026-10-30 22:57:28.159982615 +0000 UTC m=+3868.664654567"
Oct 30 22:57:34.449412 ip-10-0-131-202 kubenswrapper[4412]: E1030 22:57:34.449376    4412 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bkhmz" containerName="extractor"
Oct 30 22:58:04.448357 ip-10-0-131-202 kubenswrapper[4412]: E1030 22:58:04.448318    4412 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bkhmz" containerName="extractor"
Oct 30 22:58:33.277684 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:58:33.277653    4412 generic.go:356] "Generic (PLEG): container finished" podID="f4417fed-9bde-4318-a027-f77843fce93b" containerID="e20c295c68050d4928df195cbcf1d1ee9b373192519f5190217b2347215e2c45" exitCode=0
Oct 30 22:58:33.278139 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:58:33.277727    4412 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-4qnp4/must-gather-4xjh4" event={"ID":"f4417fed-9bde-4318-a027-f77843fce93b","Type":"ContainerDied","Data":"e20c295c68050d4928df195cbcf1d1ee9b373192519f5190217b2347215e2c45"}
Oct 30 22:58:33.278139 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:58:33.278030    4412 scope.go:122] "RemoveContainer" containerID="e20c295c68050d4928df195cbcf1d1ee9b373192519f5190217b2347215e2c45"
Oct 30 22:58:33.489980 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:58:33.489948    4412 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-4qnp4_must-gather-4xjh4_f4417fed-9bde-4318-a027-f77843fce93b/gather/0.log"
Oct 30 22:58:34.447809 ip-10-0-131-202 kubenswrapper[4412]: E1030 22:58:34.447776    4412 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bkhmz" containerName="extractor"
Oct 30 22:58:39.878792 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:58:39.878753    4412 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-must-gather-4qnp4/must-gather-4xjh4"]
Oct 30 22:58:39.879394 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:58:39.878957    4412 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-must-gather-4qnp4/must-gather-4xjh4" podUID="f4417fed-9bde-4318-a027-f77843fce93b" containerName="copy" containerID="cri-o://94fa88a98a55e90b474871b925d71c307ea4c2da7d55564a2a14420579fa294d" gracePeriod=2
Oct 30 22:58:39.929317 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:58:39.929287    4412 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-must-gather-4qnp4/must-gather-4xjh4"]
Oct 30 22:58:39.930937 ip-10-0-131-202 kubenswrapper[4412]: E1030 22:58:39.930908    4412 status_manager.go:1045] "Failed to get status for pod" err="pods \"must-gather-4xjh4\" is forbidden: User \"system:node:ip-10-0-131-202.us-west-1.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-must-gather-4qnp4\": no relationship found between node 'ip-10-0-131-202.us-west-1.compute.internal' and this object" podUID="f4417fed-9bde-4318-a027-f77843fce93b" pod="openshift-must-gather-4qnp4/must-gather-4xjh4"
Oct 30 22:58:40.158328 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:58:40.158304    4412 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-4qnp4_must-gather-4xjh4_f4417fed-9bde-4318-a027-f77843fce93b/copy/0.log"
Oct 30 22:58:40.160936 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:58:40.160918    4412 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-4qnp4/must-gather-4xjh4"
Oct 30 22:58:40.179975 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:58:40.179951    4412 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f4417fed-9bde-4318-a027-f77843fce93b-must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/f4417fed-9bde-4318-a027-f77843fce93b-must-gather-output\") pod \"f4417fed-9bde-4318-a027-f77843fce93b\" (UID: \"f4417fed-9bde-4318-a027-f77843fce93b\") "
Oct 30 22:58:40.180094 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:58:40.180002    4412 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f4417fed-9bde-4318-a027-f77843fce93b-kube-api-access-gm7h6\" (UniqueName: \"kubernetes.io/projected/f4417fed-9bde-4318-a027-f77843fce93b-kube-api-access-gm7h6\") pod \"f4417fed-9bde-4318-a027-f77843fce93b\" (UID: \"f4417fed-9bde-4318-a027-f77843fce93b\") "
Oct 30 22:58:40.181244 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:58:40.181220    4412 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f4417fed-9bde-4318-a027-f77843fce93b-must-gather-output" pod "f4417fed-9bde-4318-a027-f77843fce93b" (UID: "f4417fed-9bde-4318-a027-f77843fce93b"). InnerVolumeSpecName "must-gather-output". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:58:40.182249 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:58:40.182228    4412 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f4417fed-9bde-4318-a027-f77843fce93b-kube-api-access-gm7h6" pod "f4417fed-9bde-4318-a027-f77843fce93b" (UID: "f4417fed-9bde-4318-a027-f77843fce93b"). InnerVolumeSpecName "kube-api-access-gm7h6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:58:40.212407 ip-10-0-131-202 kubenswrapper[4412]: E1030 22:58:40.212382    4412 status_manager.go:1045] "Failed to get status for pod" err="pods \"must-gather-4xjh4\" is forbidden: User \"system:node:ip-10-0-131-202.us-west-1.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-must-gather-4qnp4\": no relationship found between node 'ip-10-0-131-202.us-west-1.compute.internal' and this object" podUID="f4417fed-9bde-4318-a027-f77843fce93b" pod="openshift-must-gather-4qnp4/must-gather-4xjh4"
Oct 30 22:58:40.281417 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:58:40.281392    4412 reconciler_common.go:299] "Volume detached for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/f4417fed-9bde-4318-a027-f77843fce93b-must-gather-output\") on node \"ip-10-0-131-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:58:40.281417 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:58:40.281414    4412 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gm7h6\" (UniqueName: \"kubernetes.io/projected/f4417fed-9bde-4318-a027-f77843fce93b-kube-api-access-gm7h6\") on node \"ip-10-0-131-202.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 22:58:40.303510 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:58:40.303490    4412 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-4qnp4_must-gather-4xjh4_f4417fed-9bde-4318-a027-f77843fce93b/copy/0.log"
Oct 30 22:58:40.303862 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:58:40.303840    4412 generic.go:356] "Generic (PLEG): container finished" podID="f4417fed-9bde-4318-a027-f77843fce93b" containerID="94fa88a98a55e90b474871b925d71c307ea4c2da7d55564a2a14420579fa294d" exitCode=143
Oct 30 22:58:40.303912 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:58:40.303896    4412 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-4qnp4/must-gather-4xjh4"
Oct 30 22:58:40.303944 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:58:40.303922    4412 scope.go:122] "RemoveContainer" containerID="94fa88a98a55e90b474871b925d71c307ea4c2da7d55564a2a14420579fa294d"
Oct 30 22:58:40.313944 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:58:40.313926    4412 scope.go:122] "RemoveContainer" containerID="e20c295c68050d4928df195cbcf1d1ee9b373192519f5190217b2347215e2c45"
Oct 30 22:58:40.326577 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:58:40.326551    4412 scope.go:122] "RemoveContainer" containerID="94fa88a98a55e90b474871b925d71c307ea4c2da7d55564a2a14420579fa294d"
Oct 30 22:58:40.326825 ip-10-0-131-202 kubenswrapper[4412]: E1030 22:58:40.326807    4412 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"94fa88a98a55e90b474871b925d71c307ea4c2da7d55564a2a14420579fa294d\": container with ID starting with 94fa88a98a55e90b474871b925d71c307ea4c2da7d55564a2a14420579fa294d not found: ID does not exist" containerID="94fa88a98a55e90b474871b925d71c307ea4c2da7d55564a2a14420579fa294d"
Oct 30 22:58:40.326881 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:58:40.326834    4412 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"94fa88a98a55e90b474871b925d71c307ea4c2da7d55564a2a14420579fa294d"} err="failed to get container status \"94fa88a98a55e90b474871b925d71c307ea4c2da7d55564a2a14420579fa294d\": rpc error: code = NotFound desc = could not find container \"94fa88a98a55e90b474871b925d71c307ea4c2da7d55564a2a14420579fa294d\": container with ID starting with 94fa88a98a55e90b474871b925d71c307ea4c2da7d55564a2a14420579fa294d not found: ID does not exist"
Oct 30 22:58:40.326881 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:58:40.326853    4412 scope.go:122] "RemoveContainer" containerID="e20c295c68050d4928df195cbcf1d1ee9b373192519f5190217b2347215e2c45"
Oct 30 22:58:40.327097 ip-10-0-131-202 kubenswrapper[4412]: E1030 22:58:40.327079    4412 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e20c295c68050d4928df195cbcf1d1ee9b373192519f5190217b2347215e2c45\": container with ID starting with e20c295c68050d4928df195cbcf1d1ee9b373192519f5190217b2347215e2c45 not found: ID does not exist" containerID="e20c295c68050d4928df195cbcf1d1ee9b373192519f5190217b2347215e2c45"
Oct 30 22:58:40.327198 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:58:40.327108    4412 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e20c295c68050d4928df195cbcf1d1ee9b373192519f5190217b2347215e2c45"} err="failed to get container status \"e20c295c68050d4928df195cbcf1d1ee9b373192519f5190217b2347215e2c45\": rpc error: code = NotFound desc = could not find container \"e20c295c68050d4928df195cbcf1d1ee9b373192519f5190217b2347215e2c45\": container with ID starting with e20c295c68050d4928df195cbcf1d1ee9b373192519f5190217b2347215e2c45 not found: ID does not exist"
Oct 30 22:58:40.355318 ip-10-0-131-202 kubenswrapper[4412]: E1030 22:58:40.355286    4412 status_manager.go:1045] "Failed to get status for pod" err="pods \"must-gather-4xjh4\" is forbidden: User \"system:node:ip-10-0-131-202.us-west-1.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-must-gather-4qnp4\": no relationship found between node 'ip-10-0-131-202.us-west-1.compute.internal' and this object" podUID="f4417fed-9bde-4318-a027-f77843fce93b" pod="openshift-must-gather-4qnp4/must-gather-4xjh4"
Oct 30 22:58:40.406695 ip-10-0-131-202 kubenswrapper[4412]: E1030 22:58:40.406630    4412 status_manager.go:1045] "Failed to get status for pod" err="pods \"must-gather-4xjh4\" is forbidden: User \"system:node:ip-10-0-131-202.us-west-1.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-must-gather-4qnp4\": no relationship found between node 'ip-10-0-131-202.us-west-1.compute.internal' and this object" podUID="f4417fed-9bde-4318-a027-f77843fce93b" pod="openshift-must-gather-4qnp4/must-gather-4xjh4"
Oct 30 22:58:41.626263 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:58:41.626233    4412 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f4417fed-9bde-4318-a027-f77843fce93b" path="/var/lib/kubelet/pods/f4417fed-9bde-4318-a027-f77843fce93b/volumes"
Oct 30 22:58:44.717393 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:58:44.717360    4412 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-4vtdc_d17b7754-d1d4-4952-97c7-7a04c714b48c/global-pull-secret-syncer/0.log"
Oct 30 22:58:48.259471 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:58:48.259443    4412 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-x7l5l_25590e1f-99e3-4469-8a69-39cd30317c28/konnectivity-agent/0.log"
Oct 30 22:58:49.284201 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:58:49.284172    4412 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-131-202.us-west-1.compute.internal_04337a6f3666f0af07c3d2b702c6e379/haproxy/0.log"
Oct 30 22:58:54.264452 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:58:54.264422    4412 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_02d2dde4-4ed7-4b5a-b639-f54c97ba473c/alertmanager/0.log"
Oct 30 22:58:54.687424 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:58:54.687342    4412 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_02d2dde4-4ed7-4b5a-b639-f54c97ba473c/config-reloader/0.log"
Oct 30 22:58:55.123982 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:58:55.123951    4412 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_02d2dde4-4ed7-4b5a-b639-f54c97ba473c/kube-rbac-proxy-web/0.log"
Oct 30 22:58:55.575044 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:58:55.575015    4412 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_02d2dde4-4ed7-4b5a-b639-f54c97ba473c/kube-rbac-proxy/0.log"
Oct 30 22:58:56.035007 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:58:56.034984    4412 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_02d2dde4-4ed7-4b5a-b639-f54c97ba473c/kube-rbac-proxy-metric/0.log"
Oct 30 22:58:56.474796 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:58:56.474774    4412 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_02d2dde4-4ed7-4b5a-b639-f54c97ba473c/prom-label-proxy/0.log"
Oct 30 22:58:56.896149 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:58:56.896046    4412 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_02d2dde4-4ed7-4b5a-b639-f54c97ba473c/init-config-reloader/0.log"
Oct 30 22:58:57.396502 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:58:57.396472    4412 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-must-gather-k2tk6/perf-node-gather-daemonset-s2xqh"]
Oct 30 22:58:57.396754 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:58:57.396743    4412 cpu_manager.go:414] "containerMap: removing container" podUID="f4417fed-9bde-4318-a027-f77843fce93b" containerName="gather"
Oct 30 22:58:57.396798 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:58:57.396757    4412 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f4417fed-9bde-4318-a027-f77843fce93b" containerName="gather"
Oct 30 22:58:57.396798 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:58:57.396765    4412 cpu_manager.go:414] "containerMap: removing container" podUID="f4417fed-9bde-4318-a027-f77843fce93b" containerName="copy"
Oct 30 22:58:57.396798 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:58:57.396771    4412 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f4417fed-9bde-4318-a027-f77843fce93b" containerName="copy"
Oct 30 22:58:57.396894 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:58:57.396829    4412 memory_manager.go:365] "RemoveStaleState removing state" podUID="f4417fed-9bde-4318-a027-f77843fce93b" containerName="gather"
Oct 30 22:58:57.396894 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:58:57.396836    4412 memory_manager.go:365] "RemoveStaleState removing state" podUID="f4417fed-9bde-4318-a027-f77843fce93b" containerName="copy"
Oct 30 22:58:57.401722 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:58:57.401704    4412 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-k2tk6/perf-node-gather-daemonset-s2xqh"
Oct 30 22:58:57.454165 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:58:57.454114    4412 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-k2tk6\"/\"openshift-service-ca.crt\""
Oct 30 22:58:57.454291 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:58:57.454176    4412 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-k2tk6\"/\"default-dockercfg-fqshl\""
Oct 30 22:58:57.454291 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:58:57.454187    4412 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-k2tk6\"/\"kube-root-ca.crt\""
Oct 30 22:58:57.491009 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:58:57.490986    4412 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-k2tk6/perf-node-gather-daemonset-s2xqh"]
Oct 30 22:58:57.514855 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:58:57.514829    4412 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/0553c9d0-f32a-41da-bc6c-e0ae6d56a964-lib-modules\") pod \"perf-node-gather-daemonset-s2xqh\" (UID: \"0553c9d0-f32a-41da-bc6c-e0ae6d56a964\") " pod="openshift-must-gather-k2tk6/perf-node-gather-daemonset-s2xqh"
Oct 30 22:58:57.514953 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:58:57.514890    4412 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0553c9d0-f32a-41da-bc6c-e0ae6d56a964-sys\") pod \"perf-node-gather-daemonset-s2xqh\" (UID: \"0553c9d0-f32a-41da-bc6c-e0ae6d56a964\") " pod="openshift-must-gather-k2tk6/perf-node-gather-daemonset-s2xqh"
Oct 30 22:58:57.515042 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:58:57.515021    4412 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dkflh\" (UniqueName: \"kubernetes.io/projected/0553c9d0-f32a-41da-bc6c-e0ae6d56a964-kube-api-access-dkflh\") pod \"perf-node-gather-daemonset-s2xqh\" (UID: \"0553c9d0-f32a-41da-bc6c-e0ae6d56a964\") " pod="openshift-must-gather-k2tk6/perf-node-gather-daemonset-s2xqh"
Oct 30 22:58:57.515085 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:58:57.515058    4412 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/0553c9d0-f32a-41da-bc6c-e0ae6d56a964-podres\") pod \"perf-node-gather-daemonset-s2xqh\" (UID: \"0553c9d0-f32a-41da-bc6c-e0ae6d56a964\") " pod="openshift-must-gather-k2tk6/perf-node-gather-daemonset-s2xqh"
Oct 30 22:58:57.515176 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:58:57.515160    4412 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/0553c9d0-f32a-41da-bc6c-e0ae6d56a964-proc\") pod \"perf-node-gather-daemonset-s2xqh\" (UID: \"0553c9d0-f32a-41da-bc6c-e0ae6d56a964\") " pod="openshift-must-gather-k2tk6/perf-node-gather-daemonset-s2xqh"
Oct 30 22:58:57.615500 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:58:57.615474    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/0553c9d0-f32a-41da-bc6c-e0ae6d56a964-proc\") pod \"perf-node-gather-daemonset-s2xqh\" (UID: \"0553c9d0-f32a-41da-bc6c-e0ae6d56a964\") " pod="openshift-must-gather-k2tk6/perf-node-gather-daemonset-s2xqh"
Oct 30 22:58:57.615625 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:58:57.615507    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/0553c9d0-f32a-41da-bc6c-e0ae6d56a964-lib-modules\") pod \"perf-node-gather-daemonset-s2xqh\" (UID: \"0553c9d0-f32a-41da-bc6c-e0ae6d56a964\") " pod="openshift-must-gather-k2tk6/perf-node-gather-daemonset-s2xqh"
Oct 30 22:58:57.615625 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:58:57.615525    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0553c9d0-f32a-41da-bc6c-e0ae6d56a964-sys\") pod \"perf-node-gather-daemonset-s2xqh\" (UID: \"0553c9d0-f32a-41da-bc6c-e0ae6d56a964\") " pod="openshift-must-gather-k2tk6/perf-node-gather-daemonset-s2xqh"
Oct 30 22:58:57.615625 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:58:57.615567    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dkflh\" (UniqueName: \"kubernetes.io/projected/0553c9d0-f32a-41da-bc6c-e0ae6d56a964-kube-api-access-dkflh\") pod \"perf-node-gather-daemonset-s2xqh\" (UID: \"0553c9d0-f32a-41da-bc6c-e0ae6d56a964\") " pod="openshift-must-gather-k2tk6/perf-node-gather-daemonset-s2xqh"
Oct 30 22:58:57.615625 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:58:57.615590    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/0553c9d0-f32a-41da-bc6c-e0ae6d56a964-proc\") pod \"perf-node-gather-daemonset-s2xqh\" (UID: \"0553c9d0-f32a-41da-bc6c-e0ae6d56a964\") " pod="openshift-must-gather-k2tk6/perf-node-gather-daemonset-s2xqh"
Oct 30 22:58:57.615625 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:58:57.615612    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/0553c9d0-f32a-41da-bc6c-e0ae6d56a964-lib-modules\") pod \"perf-node-gather-daemonset-s2xqh\" (UID: \"0553c9d0-f32a-41da-bc6c-e0ae6d56a964\") " pod="openshift-must-gather-k2tk6/perf-node-gather-daemonset-s2xqh"
Oct 30 22:58:57.615883 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:58:57.615597    4412 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/0553c9d0-f32a-41da-bc6c-e0ae6d56a964-podres\") pod \"perf-node-gather-daemonset-s2xqh\" (UID: \"0553c9d0-f32a-41da-bc6c-e0ae6d56a964\") " pod="openshift-must-gather-k2tk6/perf-node-gather-daemonset-s2xqh"
Oct 30 22:58:57.615883 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:58:57.615639    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0553c9d0-f32a-41da-bc6c-e0ae6d56a964-sys\") pod \"perf-node-gather-daemonset-s2xqh\" (UID: \"0553c9d0-f32a-41da-bc6c-e0ae6d56a964\") " pod="openshift-must-gather-k2tk6/perf-node-gather-daemonset-s2xqh"
Oct 30 22:58:57.615883 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:58:57.615654    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/0553c9d0-f32a-41da-bc6c-e0ae6d56a964-podres\") pod \"perf-node-gather-daemonset-s2xqh\" (UID: \"0553c9d0-f32a-41da-bc6c-e0ae6d56a964\") " pod="openshift-must-gather-k2tk6/perf-node-gather-daemonset-s2xqh"
Oct 30 22:58:57.674211 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:58:57.674153    4412 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dkflh\" (UniqueName: \"kubernetes.io/projected/0553c9d0-f32a-41da-bc6c-e0ae6d56a964-kube-api-access-dkflh\") pod \"perf-node-gather-daemonset-s2xqh\" (UID: \"0553c9d0-f32a-41da-bc6c-e0ae6d56a964\") " pod="openshift-must-gather-k2tk6/perf-node-gather-daemonset-s2xqh"
Oct 30 22:58:57.712827 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:58:57.712804    4412 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-k2tk6/perf-node-gather-daemonset-s2xqh"
Oct 30 22:58:58.087421 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:58:58.087399    4412 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-k2tk6/perf-node-gather-daemonset-s2xqh"]
Oct 30 22:58:58.090151 ip-10-0-131-202 kubenswrapper[4412]: W1030 22:58:58.090109    4412 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod0553c9d0_f32a_41da_bc6c_e0ae6d56a964.slice/crio-6b75f134d4c7086c46ac4abe629b70ec08753ab67178ca5c6b130e73ba56c00f WatchSource:0}: Error finding container 6b75f134d4c7086c46ac4abe629b70ec08753ab67178ca5c6b130e73ba56c00f: Status 404 returned error can't find the container with id 6b75f134d4c7086c46ac4abe629b70ec08753ab67178ca5c6b130e73ba56c00f
Oct 30 22:58:58.377024 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:58:58.376930    4412 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-k2tk6/perf-node-gather-daemonset-s2xqh" event={"ID":"0553c9d0-f32a-41da-bc6c-e0ae6d56a964","Type":"ContainerStarted","Data":"5e5b076e6bbf6e97b8be0128ae749f2368295bb5045db28adf3533f09c20205f"}
Oct 30 22:58:58.377225 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:58:58.377021    4412 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-k2tk6/perf-node-gather-daemonset-s2xqh" event={"ID":"0553c9d0-f32a-41da-bc6c-e0ae6d56a964","Type":"ContainerStarted","Data":"6b75f134d4c7086c46ac4abe629b70ec08753ab67178ca5c6b130e73ba56c00f"}
Oct 30 22:58:58.377225 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:58:58.377053    4412 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-k2tk6/perf-node-gather-daemonset-s2xqh"
Oct 30 22:58:58.492511 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:58:58.492463    4412 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-must-gather-k2tk6/perf-node-gather-daemonset-s2xqh" podStartSLOduration=1.492448102 podStartE2EDuration="1.492448102s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 22:58:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:58:58.492091023 +0000 UTC m=+3958.996762987" watchObservedRunningTime="2026-10-30 22:58:58.492448102 +0000 UTC m=+3958.997120045"
Oct 30 22:58:59.852737 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:58:59.852709    4412 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-ckb8k_78d5206b-1dec-4c66-97e1-32b034ce3529/node-exporter/0.log"
Oct 30 22:59:00.269880 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:59:00.269855    4412 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-ckb8k_78d5206b-1dec-4c66-97e1-32b034ce3529/kube-rbac-proxy/0.log"
Oct 30 22:59:00.688023 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:59:00.687983    4412 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-ckb8k_78d5206b-1dec-4c66-97e1-32b034ce3529/init-textfile/0.log"
Oct 30 22:59:03.636140 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:59:03.636092    4412 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-7bd7489957-5gm8c_b9c973ab-4163-413a-8969-2947502f3592/kube-rbac-proxy-main/0.log"
Oct 30 22:59:04.059934 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:59:04.059909    4412 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-7bd7489957-5gm8c_b9c973ab-4163-413a-8969-2947502f3592/kube-rbac-proxy-self/0.log"
Oct 30 22:59:04.390076 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:59:04.390011    4412 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-k2tk6/perf-node-gather-daemonset-s2xqh"
Oct 30 22:59:04.448052 ip-10-0-131-202 kubenswrapper[4412]: E1030 22:59:04.448014    4412 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bkhmz" containerName="extractor"
Oct 30 22:59:04.477227 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:59:04.477202    4412 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-7bd7489957-5gm8c_b9c973ab-4163-413a-8969-2947502f3592/openshift-state-metrics/0.log"
Oct 30 22:59:04.911963 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:59:04.911918    4412 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_e2aaf8b0-d53f-4452-92f6-170dd6cb201a/prometheus/0.log"
Oct 30 22:59:05.323335 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:59:05.323305    4412 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_e2aaf8b0-d53f-4452-92f6-170dd6cb201a/config-reloader/0.log"
Oct 30 22:59:05.742787 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:59:05.742759    4412 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_e2aaf8b0-d53f-4452-92f6-170dd6cb201a/thanos-sidecar/0.log"
Oct 30 22:59:06.162584 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:59:06.162509    4412 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_e2aaf8b0-d53f-4452-92f6-170dd6cb201a/kube-rbac-proxy-web/0.log"
Oct 30 22:59:06.580997 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:59:06.580963    4412 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_e2aaf8b0-d53f-4452-92f6-170dd6cb201a/kube-rbac-proxy/0.log"
Oct 30 22:59:07.004078 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:59:07.004050    4412 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_e2aaf8b0-d53f-4452-92f6-170dd6cb201a/kube-rbac-proxy-thanos/0.log"
Oct 30 22:59:07.428352 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:59:07.428272    4412 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_e2aaf8b0-d53f-4452-92f6-170dd6cb201a/init-config-reloader/0.log"
Oct 30 22:59:07.856394 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:59:07.856366    4412 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-9cd89b74c-plp26_0886e99f-8cba-48a7-9dae-e13d46230fac/prometheus-operator/0.log"
Oct 30 22:59:08.271329 ip-10-0-131-202 kubenswrapper[4412]: I1030 22:59:08.271303    4412 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-9cd89b74c-plp26_0886e99f-8cba-48a7-9dae-e13d46230fac/kube-rbac-proxy/0.log"