Oct 30 21:53:42.314351 ip-10-0-143-246 systemd[1]: Starting Kubernetes Kubelet...
Oct 30 21:53:42.704413 ip-10-0-143-246 kubenswrapper[2577]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 30 21:53:42.704413 ip-10-0-143-246 kubenswrapper[2577]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Oct 30 21:53:42.704413 ip-10-0-143-246 kubenswrapper[2577]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 30 21:53:42.704413 ip-10-0-143-246 kubenswrapper[2577]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 30 21:53:42.709622 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.709489    2577 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 21:53:42.709622 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.709619    2577 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 21:53:42.709622 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.709625    2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 21:53:42.709622 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.709629    2577 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 21:53:42.709860 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.709634    2577 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 21:53:42.709860 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.709638    2577 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 21:53:42.709860 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.709652    2577 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 21:53:42.709860 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.709657    2577 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 21:53:42.709860 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.709661    2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 21:53:42.709860 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.709666    2577 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 21:53:42.709860 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.709671    2577 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 21:53:42.709860 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.709675    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 21:53:42.709860 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.709680    2577 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 21:53:42.709860 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.709684    2577 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 21:53:42.709860 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.709688    2577 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 21:53:42.709860 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.709692    2577 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 21:53:42.709860 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.709696    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 21:53:42.709860 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.709701    2577 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 21:53:42.709860 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.709705    2577 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 21:53:42.709860 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.709716    2577 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 21:53:42.709860 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.709721    2577 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 21:53:42.709860 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.709725    2577 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 21:53:42.709860 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.709730    2577 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 21:53:42.709860 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.709734    2577 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 21:53:42.710418 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.709739    2577 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 21:53:42.710418 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.709743    2577 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 21:53:42.710418 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.709747    2577 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 21:53:42.710418 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.709751    2577 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 21:53:42.710418 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.709756    2577 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 21:53:42.710418 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.709760    2577 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 21:53:42.710418 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.709765    2577 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 21:53:42.710418 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.709774    2577 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 21:53:42.710418 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.709778    2577 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 21:53:42.710418 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.709783    2577 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 21:53:42.710418 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.709787    2577 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 21:53:42.710418 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.709791    2577 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 21:53:42.710418 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.709796    2577 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 21:53:42.710418 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.709801    2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 21:53:42.710418 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.709806    2577 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 21:53:42.710418 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.709810    2577 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 21:53:42.710418 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.709814    2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 21:53:42.710418 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.709818    2577 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 21:53:42.710418 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.709823    2577 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 21:53:42.710873 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.709827    2577 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 21:53:42.710873 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.709836    2577 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 21:53:42.710873 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.709840    2577 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 21:53:42.710873 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.709844    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 21:53:42.710873 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.709849    2577 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 21:53:42.710873 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.709853    2577 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 21:53:42.710873 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.709857    2577 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 21:53:42.710873 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.709861    2577 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 21:53:42.710873 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.709867    2577 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 21:53:42.710873 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.709872    2577 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 21:53:42.710873 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.709877    2577 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 21:53:42.710873 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.709884    2577 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 21:53:42.710873 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.709890    2577 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 21:53:42.710873 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.709968    2577 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 21:53:42.710873 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.710156    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 21:53:42.710873 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.710164    2577 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 21:53:42.710873 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.710169    2577 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 21:53:42.710873 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.710177    2577 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 21:53:42.711323 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.710183    2577 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 21:53:42.711323 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.710188    2577 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 21:53:42.711323 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.710193    2577 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 21:53:42.711323 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.710197    2577 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 21:53:42.711323 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.710201    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 21:53:42.711323 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.710205    2577 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 21:53:42.711323 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.710210    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 21:53:42.711323 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.710213    2577 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 21:53:42.711323 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.710217    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 21:53:42.711323 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.710221    2577 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 21:53:42.711323 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.710225    2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 21:53:42.711323 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.710229    2577 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 21:53:42.711323 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.710234    2577 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 21:53:42.711323 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.710239    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 21:53:42.711323 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.710243    2577 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 21:53:42.711323 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.710248    2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 21:53:42.711323 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.710252    2577 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 21:53:42.711323 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.710256    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 21:53:42.711323 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.710260    2577 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 21:53:42.711323 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.710264    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 21:53:42.711802 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.710268    2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 21:53:42.711802 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.710273    2577 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 21:53:42.711802 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.710277    2577 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 21:53:42.711802 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.710282    2577 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 21:53:42.711802 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.710286    2577 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 21:53:42.711802 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.710290    2577 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 21:53:42.711802 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.710294    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 21:53:42.711802 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.710298    2577 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 21:53:42.711802 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.710303    2577 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 21:53:42.711802 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.710307    2577 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 21:53:42.711802 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.710311    2577 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 21:53:42.711802 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.710315    2577 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 21:53:42.711802 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.710320    2577 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 21:53:42.711802 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.710323    2577 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 21:53:42.711802 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.710327    2577 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 21:53:42.711802 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.710331    2577 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 21:53:42.711802 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.710336    2577 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 21:53:42.711802 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.710340    2577 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 21:53:42.711802 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.710344    2577 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 21:53:42.711802 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.710348    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 21:53:42.712281 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.710351    2577 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 21:53:42.712281 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.710355    2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 21:53:42.712281 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.710360    2577 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 21:53:42.712281 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.710363    2577 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 21:53:42.712281 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.710366    2577 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 21:53:42.712281 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.711210    2577 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 21:53:42.712281 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.711218    2577 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 21:53:42.712281 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.711221    2577 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 21:53:42.712281 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.711224    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 21:53:42.712281 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.711227    2577 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 21:53:42.712281 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.711230    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 21:53:42.712281 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.711233    2577 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 21:53:42.712281 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.711235    2577 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 21:53:42.712281 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.711238    2577 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 21:53:42.712281 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.711241    2577 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 21:53:42.712281 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.711243    2577 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 21:53:42.712281 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.711246    2577 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 21:53:42.712281 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.711249    2577 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 21:53:42.712281 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.711251    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 21:53:42.712744 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.711254    2577 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 21:53:42.712744 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.711257    2577 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 21:53:42.712744 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.711260    2577 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 21:53:42.712744 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.711263    2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 21:53:42.712744 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.711265    2577 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 21:53:42.712744 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.711268    2577 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 21:53:42.712744 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.711271    2577 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 21:53:42.712744 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.711275    2577 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 21:53:42.712744 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.711280    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 21:53:42.712744 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.711283    2577 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 21:53:42.712744 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.711285    2577 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 21:53:42.712744 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.711289    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 21:53:42.712744 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.711291    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 21:53:42.712744 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.711294    2577 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 21:53:42.712744 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.711297    2577 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 21:53:42.712744 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.711300    2577 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 21:53:42.712744 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.711303    2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 21:53:42.712744 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.711306    2577 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 21:53:42.712744 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.711309    2577 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 21:53:42.713213 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.711312    2577 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 21:53:42.713213 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.711314    2577 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 21:53:42.713213 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.711317    2577 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 21:53:42.713213 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.711320    2577 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 21:53:42.713213 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.711322    2577 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 21:53:42.713213 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.711325    2577 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 21:53:42.713213 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.711328    2577 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 21:53:42.713213 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.711330    2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 21:53:42.713213 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.711333    2577 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 21:53:42.713213 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.711336    2577 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 21:53:42.713213 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.711339    2577 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 21:53:42.713213 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.711341    2577 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 21:53:42.713213 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.711344    2577 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 21:53:42.713213 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.711346    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 21:53:42.713213 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.711350    2577 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 21:53:42.713213 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.711353    2577 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 21:53:42.713213 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.711355    2577 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 21:53:42.713213 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.711358    2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 21:53:42.713213 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.711360    2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 21:53:42.713213 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.711363    2577 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 21:53:42.713695 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.711365    2577 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 21:53:42.713695 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.711368    2577 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 21:53:42.713695 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.711370    2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 21:53:42.713695 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.711373    2577 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 21:53:42.713695 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.711375    2577 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 21:53:42.713695 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.711378    2577 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 21:53:42.713695 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.711380    2577 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 21:53:42.713695 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.711385    2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 21:53:42.713695 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.711388    2577 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 21:53:42.713695 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.711391    2577 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 21:53:42.713695 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.711394    2577 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 21:53:42.713695 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.711397    2577 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 21:53:42.713695 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.711399    2577 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 21:53:42.713695 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.711402    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 21:53:42.713695 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.711405    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 21:53:42.713695 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.711407    2577 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 21:53:42.713695 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.711410    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 21:53:42.713695 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.711413    2577 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 21:53:42.713695 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.711415    2577 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 21:53:42.714174 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.711418    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 21:53:42.714174 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.711420    2577 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 21:53:42.714174 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.711424    2577 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 21:53:42.714174 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.711429    2577 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 21:53:42.714174 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.711433    2577 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 21:53:42.714174 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.711436    2577 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 21:53:42.714174 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.711439    2577 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 21:53:42.714174 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.711443    2577 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 21:53:42.714174 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.711446    2577 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 21:53:42.714174 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.711449    2577 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 21:53:42.714174 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.711451    2577 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 21:53:42.714174 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.711454    2577 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 21:53:42.714174 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.711456    2577 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 21:53:42.714174 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.711459    2577 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 21:53:42.714174 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.711462    2577 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 21:53:42.714174 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.711464    2577 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 21:53:42.714174 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.711467    2577 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 21:53:42.714174 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.711470    2577 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 21:53:42.714174 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.711472    2577 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 21:53:42.714632 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.711475    2577 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 21:53:42.714632 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.711479    2577 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 21:53:42.714632 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.711482    2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 21:53:42.714632 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.711484    2577 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 21:53:42.714632 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.711487    2577 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 21:53:42.714632 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.711490    2577 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 21:53:42.714632 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.711492    2577 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 21:53:42.714632 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.711495    2577 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 21:53:42.714632 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.711498    2577 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 21:53:42.714632 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.711500    2577 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 21:53:42.714632 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.711503    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 21:53:42.714632 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.711505    2577 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 21:53:42.714632 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.711508    2577 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 21:53:42.714632 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.711510    2577 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 21:53:42.714632 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.711513    2577 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 21:53:42.714632 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.712059    2577 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 21:53:42.714632 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.712065    2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 21:53:42.714632 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.712068    2577 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 21:53:42.714632 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.712071    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 21:53:42.714632 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.712074    2577 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 21:53:42.715130 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.712077    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 21:53:42.715130 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.712082    2577 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 21:53:42.715130 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.712086    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 21:53:42.715130 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.712089    2577 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 21:53:42.715130 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.712092    2577 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 21:53:42.715130 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.712094    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 21:53:42.715130 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.712097    2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 21:53:42.715130 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.712100    2577 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 21:53:42.715130 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.712102    2577 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 21:53:42.715130 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.712105    2577 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 21:53:42.715130 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.712107    2577 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 21:53:42.715130 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.712110    2577 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 21:53:42.715130 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.712112    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 21:53:42.715130 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.712116    2577 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 21:53:42.715130 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.712119    2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 21:53:42.715130 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.712121    2577 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 21:53:42.715130 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.712124    2577 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 21:53:42.715130 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.712127    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 21:53:42.715130 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.712129    2577 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 21:53:42.715590 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.712132    2577 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 21:53:42.715590 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.712134    2577 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 21:53:42.715590 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.712137    2577 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 21:53:42.715590 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.712139    2577 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 21:53:42.715590 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.712141    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 21:53:42.715590 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.712145    2577 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 21:53:42.715590 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.712147    2577 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 21:53:42.715590 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.712150    2577 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 21:53:42.715590 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.712152    2577 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 21:53:42.715590 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.712155    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 21:53:42.715590 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.712157    2577 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 21:53:42.715590 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.712160    2577 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 21:53:42.715590 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.712163    2577 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 21:53:42.715590 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.712166    2577 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 21:53:42.715590 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.712169    2577 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 21:53:42.715590 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.712171    2577 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 21:53:42.715590 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.712174    2577 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 21:53:42.715590 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.712177    2577 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 21:53:42.715590 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.712179    2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 21:53:42.715590 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.712182    2577 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 21:53:42.716095 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.712184    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 21:53:42.716095 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.712193    2577 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 21:53:42.716095 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.712195    2577 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 21:53:42.716095 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.712198    2577 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 21:53:42.716095 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.712201    2577 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 21:53:42.716095 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.712204    2577 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 21:53:42.716095 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.712206    2577 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 21:53:42.716095 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.712209    2577 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 21:53:42.716095 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.712211    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 21:53:42.716095 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.712214    2577 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 21:53:42.716095 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.712216    2577 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 21:53:42.716095 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.712219    2577 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 21:53:42.716095 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.712221    2577 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 21:53:42.716095 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.712224    2577 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 21:53:42.716095 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.712226    2577 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 21:53:42.716095 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.712229    2577 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 21:53:42.716095 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.712231    2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 21:53:42.716095 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.712234    2577 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 21:53:42.716095 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.712237    2577 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 21:53:42.716095 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.712239    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 21:53:42.716583 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.712242    2577 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 21:53:42.716583 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.712244    2577 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 21:53:42.716583 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.712246    2577 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 21:53:42.716583 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.712249    2577 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 21:53:42.716583 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.712252    2577 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 21:53:42.716583 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.712257    2577 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 21:53:42.716583 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.712259    2577 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 21:53:42.716583 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.712262    2577 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 21:53:42.716583 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.712265    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 21:53:42.716583 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.712268    2577 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 21:53:42.716583 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.712270    2577 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 21:53:42.716583 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.712273    2577 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 21:53:42.716583 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.712276    2577 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 21:53:42.716583 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.712279    2577 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 21:53:42.716583 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.712281    2577 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 21:53:42.716583 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.712284    2577 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 21:53:42.716583 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.712287    2577 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 21:53:42.716583 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.712289    2577 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 21:53:42.716583 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.712292    2577 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 21:53:42.717044 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.712296    2577 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 21:53:42.717044 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.712298    2577 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 21:53:42.717044 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.712301    2577 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 21:53:42.717044 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.712305    2577 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 21:53:42.717044 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.712309    2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 21:53:42.717044 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.712313    2577 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 21:53:42.717044 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.712315    2577 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 21:53:42.717044 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.712318    2577 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 21:53:42.717044 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.712321    2577 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 21:53:42.717044 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.712323    2577 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 21:53:42.717044 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.712326    2577 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 21:53:42.717044 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.712329    2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 21:53:42.717044 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.712331    2577 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 21:53:42.717044 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.712334    2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 21:53:42.717044 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.712336    2577 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 21:53:42.717044 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.712339    2577 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 21:53:42.717044 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.712341    2577 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 21:53:42.717044 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.712344    2577 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 21:53:42.717044 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.712350    2577 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 21:53:42.717594 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.712353    2577 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 21:53:42.717594 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.712355    2577 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 21:53:42.717594 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.712358    2577 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 21:53:42.717594 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.712360    2577 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 21:53:42.717594 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:42.713251    2577 flags.go:64] FLAG: --address="0.0.0.0"
Oct 30 21:53:42.717594 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:42.713262    2577 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Oct 30 21:53:42.717594 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:42.713270    2577 flags.go:64] FLAG: --anonymous-auth="true"
Oct 30 21:53:42.717594 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:42.713275    2577 flags.go:64] FLAG: --application-metrics-count-limit="100"
Oct 30 21:53:42.717594 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:42.713280    2577 flags.go:64] FLAG: --authentication-token-webhook="false"
Oct 30 21:53:42.717594 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:42.713283    2577 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Oct 30 21:53:42.717594 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:42.713288    2577 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Oct 30 21:53:42.717594 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:42.713293    2577 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Oct 30 21:53:42.717594 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:42.713296    2577 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Oct 30 21:53:42.717594 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:42.713300    2577 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Oct 30 21:53:42.717594 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:42.713304    2577 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Oct 30 21:53:42.717594 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:42.713307    2577 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Oct 30 21:53:42.717594 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:42.713310    2577 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Oct 30 21:53:42.717594 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:42.713313    2577 flags.go:64] FLAG: --cgroup-root=""
Oct 30 21:53:42.717594 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:42.713316    2577 flags.go:64] FLAG: --cgroups-per-qos="true"
Oct 30 21:53:42.717594 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:42.713320    2577 flags.go:64] FLAG: --client-ca-file=""
Oct 30 21:53:42.717594 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:42.713323    2577 flags.go:64] FLAG: --cloud-provider="external"
Oct 30 21:53:42.717594 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:42.713327    2577 flags.go:64] FLAG: --cluster-dns="[]"
Oct 30 21:53:42.717594 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:42.713332    2577 flags.go:64] FLAG: --cluster-domain=""
Oct 30 21:53:42.717594 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:42.713335    2577 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Oct 30 21:53:42.718248 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:42.713338    2577 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Oct 30 21:53:42.718248 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:42.713341    2577 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Oct 30 21:53:42.718248 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:42.713345    2577 flags.go:64] FLAG: --container-log-max-files="5"
Oct 30 21:53:42.718248 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:42.713349    2577 flags.go:64] FLAG: --container-log-max-size="10Mi"
Oct 30 21:53:42.718248 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:42.713352    2577 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Oct 30 21:53:42.718248 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:42.713355    2577 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Oct 30 21:53:42.718248 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:42.713358    2577 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Oct 30 21:53:42.718248 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:42.713368    2577 flags.go:64] FLAG: --contention-profiling="false"
Oct 30 21:53:42.718248 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:42.713372    2577 flags.go:64] FLAG: --cpu-cfs-quota="true"
Oct 30 21:53:42.718248 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:42.713375    2577 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Oct 30 21:53:42.718248 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:42.713378    2577 flags.go:64] FLAG: --cpu-manager-policy="none"
Oct 30 21:53:42.718248 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:42.713381    2577 flags.go:64] FLAG: --cpu-manager-policy-options=""
Oct 30 21:53:42.718248 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:42.713385    2577 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Oct 30 21:53:42.718248 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:42.713391    2577 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Oct 30 21:53:42.718248 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:42.713394    2577 flags.go:64] FLAG: --enable-debugging-handlers="true"
Oct 30 21:53:42.718248 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:42.713397    2577 flags.go:64] FLAG: --enable-load-reader="false"
Oct 30 21:53:42.718248 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:42.713400    2577 flags.go:64] FLAG: --enable-server="true"
Oct 30 21:53:42.718248 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:42.713403    2577 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Oct 30 21:53:42.718248 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:42.713408    2577 flags.go:64] FLAG: --event-burst="100"
Oct 30 21:53:42.718248 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:42.713411    2577 flags.go:64] FLAG: --event-qps="50"
Oct 30 21:53:42.718248 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:42.713414    2577 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Oct 30 21:53:42.718248 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:42.713417    2577 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Oct 30 21:53:42.718248 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:42.713420    2577 flags.go:64] FLAG: --eviction-hard=""
Oct 30 21:53:42.718248 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:42.713425    2577 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Oct 30 21:53:42.718862 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:42.713428    2577 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Oct 30 21:53:42.718862 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:42.713431    2577 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Oct 30 21:53:42.718862 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:42.713434    2577 flags.go:64] FLAG: --eviction-soft=""
Oct 30 21:53:42.718862 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:42.713437    2577 flags.go:64] FLAG: --eviction-soft-grace-period=""
Oct 30 21:53:42.718862 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:42.713441    2577 flags.go:64] FLAG: --exit-on-lock-contention="false"
Oct 30 21:53:42.718862 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:42.713444    2577 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Oct 30 21:53:42.718862 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:42.713447    2577 flags.go:64] FLAG: --experimental-mounter-path=""
Oct 30 21:53:42.718862 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:42.713450    2577 flags.go:64] FLAG: --fail-cgroupv1="true"
Oct 30 21:53:42.718862 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:42.713453    2577 flags.go:64] FLAG: --fail-swap-on="true"
Oct 30 21:53:42.718862 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:42.713456    2577 flags.go:64] FLAG: --feature-gates=""
Oct 30 21:53:42.718862 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:42.713463    2577 flags.go:64] FLAG: --file-check-frequency="20s"
Oct 30 21:53:42.718862 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:42.713467    2577 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Oct 30 21:53:42.718862 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:42.713470    2577 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Oct 30 21:53:42.718862 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:42.713473    2577 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Oct 30 21:53:42.718862 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:42.713477    2577 flags.go:64] FLAG: --healthz-port="10248"
Oct 30 21:53:42.718862 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:42.713481    2577 flags.go:64] FLAG: --help="false"
Oct 30 21:53:42.718862 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:42.713484    2577 flags.go:64] FLAG: --hostname-override="ip-10-0-143-246.ec2.internal"
Oct 30 21:53:42.718862 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:42.713490    2577 flags.go:64] FLAG: --housekeeping-interval="10s"
Oct 30 21:53:42.718862 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:42.713493    2577 flags.go:64] FLAG: --http-check-frequency="20s"
Oct 30 21:53:42.718862 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:42.713496    2577 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Oct 30 21:53:42.718862 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:42.713500    2577 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Oct 30 21:53:42.718862 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:42.713504    2577 flags.go:64] FLAG: --image-gc-high-threshold="85"
Oct 30 21:53:42.718862 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:42.713508    2577 flags.go:64] FLAG: --image-gc-low-threshold="80"
Oct 30 21:53:42.718862 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:42.713511    2577 flags.go:64] FLAG: --image-service-endpoint=""
Oct 30 21:53:42.719475 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:42.713514    2577 flags.go:64] FLAG: --kernel-memcg-notification="false"
Oct 30 21:53:42.719475 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:42.713517    2577 flags.go:64] FLAG: --kube-api-burst="100"
Oct 30 21:53:42.719475 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:42.713520    2577 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Oct 30 21:53:42.719475 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:42.713523    2577 flags.go:64] FLAG: --kube-api-qps="50"
Oct 30 21:53:42.719475 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:42.713526    2577 flags.go:64] FLAG: --kube-reserved=""
Oct 30 21:53:42.719475 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:42.713529    2577 flags.go:64] FLAG: --kube-reserved-cgroup=""
Oct 30 21:53:42.719475 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:42.713532    2577 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Oct 30 21:53:42.719475 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:42.713535    2577 flags.go:64] FLAG: --kubelet-cgroups=""
Oct 30 21:53:42.719475 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:42.713538    2577 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Oct 30 21:53:42.719475 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:42.713541    2577 flags.go:64] FLAG: --lock-file=""
Oct 30 21:53:42.719475 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:42.713543    2577 flags.go:64] FLAG: --log-cadvisor-usage="false"
Oct 30 21:53:42.719475 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:42.713546    2577 flags.go:64] FLAG: --log-flush-frequency="5s"
Oct 30 21:53:42.719475 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:42.713549    2577 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Oct 30 21:53:42.719475 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:42.713555    2577 flags.go:64] FLAG: --log-json-split-stream="false"
Oct 30 21:53:42.719475 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:42.713558    2577 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Oct 30 21:53:42.719475 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:42.713561    2577 flags.go:64] FLAG: --log-text-split-stream="false"
Oct 30 21:53:42.719475 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:42.713564    2577 flags.go:64] FLAG: --logging-format="text"
Oct 30 21:53:42.719475 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:42.713567    2577 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Oct 30 21:53:42.719475 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:42.713571    2577 flags.go:64] FLAG: --make-iptables-util-chains="true"
Oct 30 21:53:42.719475 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:42.713574    2577 flags.go:64] FLAG: --manifest-url=""
Oct 30 21:53:42.719475 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:42.713577    2577 flags.go:64] FLAG: --manifest-url-header=""
Oct 30 21:53:42.719475 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:42.713582    2577 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Oct 30 21:53:42.719475 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:42.713585    2577 flags.go:64] FLAG: --max-open-files="1000000"
Oct 30 21:53:42.719475 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:42.713589    2577 flags.go:64] FLAG: --max-pods="110"
Oct 30 21:53:42.719475 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:42.713592    2577 flags.go:64] FLAG: --maximum-dead-containers="-1"
Oct 30 21:53:42.720130 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:42.713595    2577 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Oct 30 21:53:42.720130 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:42.713599    2577 flags.go:64] FLAG: --memory-manager-policy="None"
Oct 30 21:53:42.720130 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:42.713602    2577 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Oct 30 21:53:42.720130 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:42.713605    2577 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Oct 30 21:53:42.720130 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:42.713610    2577 flags.go:64] FLAG: --node-ip="0.0.0.0"
Oct 30 21:53:42.720130 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:42.713613    2577 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Oct 30 21:53:42.720130 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:42.713621    2577 flags.go:64] FLAG: --node-status-max-images="50"
Oct 30 21:53:42.720130 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:42.713624    2577 flags.go:64] FLAG: --node-status-update-frequency="10s"
Oct 30 21:53:42.720130 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:42.713627    2577 flags.go:64] FLAG: --oom-score-adj="-999"
Oct 30 21:53:42.720130 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:42.713630    2577 flags.go:64] FLAG: --pod-cidr=""
Oct 30 21:53:42.720130 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:42.713633    2577 flags.go:64] FLAG: --pod-manifest-path=""
Oct 30 21:53:42.720130 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:42.713636    2577 flags.go:64] FLAG: --pod-max-pids="-1"
Oct 30 21:53:42.720130 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:42.713640    2577 flags.go:64] FLAG: --pods-per-core="0"
Oct 30 21:53:42.720130 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:42.713643    2577 flags.go:64] FLAG: --port="10250"
Oct 30 21:53:42.720130 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:42.713646    2577 flags.go:64] FLAG: --protect-kernel-defaults="false"
Oct 30 21:53:42.720130 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:42.713649    2577 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0134d5967aa4bbd7b"
Oct 30 21:53:42.720130 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:42.713652    2577 flags.go:64] FLAG: --qos-reserved=""
Oct 30 21:53:42.720130 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:42.713655    2577 flags.go:64] FLAG: --read-only-port="10255"
Oct 30 21:53:42.720130 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:42.713659    2577 flags.go:64] FLAG: --register-node="true"
Oct 30 21:53:42.720130 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:42.713662    2577 flags.go:64] FLAG: --register-with-taints=""
Oct 30 21:53:42.720130 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:42.713666    2577 flags.go:64] FLAG: --registry-burst="10"
Oct 30 21:53:42.720130 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:42.713669    2577 flags.go:64] FLAG: --registry-qps="5"
Oct 30 21:53:42.720130 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:42.713672    2577 flags.go:64] FLAG: --reserved-cpus=""
Oct 30 21:53:42.720130 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:42.713675    2577 flags.go:64] FLAG: --reserved-memory=""
Oct 30 21:53:42.720130 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:42.713680    2577 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Oct 30 21:53:42.720130 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:42.713683    2577 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Oct 30 21:53:42.720753 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:42.713686    2577 flags.go:64] FLAG: --rotate-certificates="false"
Oct 30 21:53:42.720753 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:42.713689    2577 flags.go:64] FLAG: --rotate-server-certificates="false"
Oct 30 21:53:42.720753 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:42.713692    2577 flags.go:64] FLAG: --runonce="false"
Oct 30 21:53:42.720753 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:42.713695    2577 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Oct 30 21:53:42.720753 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:42.713699    2577 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Oct 30 21:53:42.720753 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:42.713702    2577 flags.go:64] FLAG: --seccomp-default="false"
Oct 30 21:53:42.720753 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:42.713705    2577 flags.go:64] FLAG: --serialize-image-pulls="true"
Oct 30 21:53:42.720753 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:42.713708    2577 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Oct 30 21:53:42.720753 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:42.713712    2577 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Oct 30 21:53:42.720753 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:42.713715    2577 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Oct 30 21:53:42.720753 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:42.713721    2577 flags.go:64] FLAG: --storage-driver-password="root"
Oct 30 21:53:42.720753 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:42.713724    2577 flags.go:64] FLAG: --storage-driver-secure="false"
Oct 30 21:53:42.720753 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:42.713727    2577 flags.go:64] FLAG: --storage-driver-table="stats"
Oct 30 21:53:42.720753 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:42.713730    2577 flags.go:64] FLAG: --storage-driver-user="root"
Oct 30 21:53:42.720753 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:42.713733    2577 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Oct 30 21:53:42.720753 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:42.713736    2577 flags.go:64] FLAG: --sync-frequency="1m0s"
Oct 30 21:53:42.720753 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:42.713740    2577 flags.go:64] FLAG: --system-cgroups=""
Oct 30 21:53:42.720753 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:42.713742    2577 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Oct 30 21:53:42.720753 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:42.713748    2577 flags.go:64] FLAG: --system-reserved-cgroup=""
Oct 30 21:53:42.720753 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:42.713752    2577 flags.go:64] FLAG: --tls-cert-file=""
Oct 30 21:53:42.720753 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:42.713755    2577 flags.go:64] FLAG: --tls-cipher-suites="[]"
Oct 30 21:53:42.720753 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:42.713762    2577 flags.go:64] FLAG: --tls-min-version=""
Oct 30 21:53:42.720753 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:42.713765    2577 flags.go:64] FLAG: --tls-private-key-file=""
Oct 30 21:53:42.720753 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:42.713768    2577 flags.go:64] FLAG: --topology-manager-policy="none"
Oct 30 21:53:42.720753 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:42.713771    2577 flags.go:64] FLAG: --topology-manager-policy-options=""
Oct 30 21:53:42.721364 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:42.713775    2577 flags.go:64] FLAG: --topology-manager-scope="container"
Oct 30 21:53:42.721364 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:42.713778    2577 flags.go:64] FLAG: --v="2"
Oct 30 21:53:42.721364 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:42.713783    2577 flags.go:64] FLAG: --version="false"
Oct 30 21:53:42.721364 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:42.713787    2577 flags.go:64] FLAG: --vmodule=""
Oct 30 21:53:42.721364 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:42.713791    2577 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Oct 30 21:53:42.721364 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:42.713794    2577 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Oct 30 21:53:42.721364 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.713938    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 21:53:42.721364 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.713942    2577 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 21:53:42.721364 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.713945    2577 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 21:53:42.721364 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.713947    2577 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 21:53:42.721364 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.713950    2577 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 21:53:42.721364 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.713953    2577 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 21:53:42.721364 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.713956    2577 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 21:53:42.721364 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.713959    2577 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 21:53:42.721364 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.713962    2577 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 21:53:42.721364 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.713964    2577 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 21:53:42.721364 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.713972    2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 21:53:42.721364 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.713977    2577 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 21:53:42.721364 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.713979    2577 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 21:53:42.721364 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.713982    2577 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 21:53:42.721364 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.714000    2577 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 21:53:42.721855 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.714005    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 21:53:42.721855 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.714009    2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 21:53:42.721855 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.714011    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 21:53:42.721855 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.714014    2577 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 21:53:42.721855 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.714017    2577 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 21:53:42.721855 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.714020    2577 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 21:53:42.721855 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.714023    2577 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 21:53:42.721855 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.714027    2577 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 21:53:42.721855 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.714030    2577 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 21:53:42.721855 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.714032    2577 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 21:53:42.721855 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.714035    2577 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 21:53:42.721855 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.714038    2577 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 21:53:42.721855 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.714041    2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 21:53:42.721855 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.714044    2577 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 21:53:42.721855 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.714046    2577 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 21:53:42.721855 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.714049    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 21:53:42.721855 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.714052    2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 21:53:42.721855 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.714055    2577 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 21:53:42.721855 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.714058    2577 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 21:53:42.721855 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.714060    2577 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 21:53:42.722434 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.714063    2577 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 21:53:42.722434 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.714065    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 21:53:42.722434 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.714069    2577 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 21:53:42.722434 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.714072    2577 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 21:53:42.722434 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.714075    2577 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 21:53:42.722434 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.714077    2577 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 21:53:42.722434 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.714080    2577 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 21:53:42.722434 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.714083    2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 21:53:42.722434 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.714089    2577 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 21:53:42.722434 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.714092    2577 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 21:53:42.722434 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.714095    2577 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 21:53:42.722434 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.714098    2577 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 21:53:42.722434 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.714100    2577 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 21:53:42.722434 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.714103    2577 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 21:53:42.722434 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.714105    2577 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 21:53:42.722434 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.714108    2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 21:53:42.722434 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.714111    2577 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 21:53:42.722434 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.714113    2577 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 21:53:42.722434 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.714116    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 21:53:42.722434 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.714120    2577 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 21:53:42.722961 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.714123    2577 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 21:53:42.722961 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.714125    2577 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 21:53:42.722961 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.714128    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 21:53:42.722961 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.714131    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 21:53:42.722961 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.714134    2577 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 21:53:42.722961 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.714137    2577 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 21:53:42.722961 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.714141    2577 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 21:53:42.722961 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.714145    2577 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 21:53:42.722961 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.714148    2577 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 21:53:42.722961 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.714150    2577 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 21:53:42.722961 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.714153    2577 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 21:53:42.722961 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.714156    2577 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 21:53:42.722961 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.714158    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 21:53:42.722961 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.714161    2577 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 21:53:42.722961 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.714164    2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 21:53:42.722961 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.714166    2577 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 21:53:42.722961 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.714169    2577 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 21:53:42.722961 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.714171    2577 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 21:53:42.722961 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.714174    2577 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 21:53:42.723439 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.714178    2577 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 21:53:42.723439 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.714183    2577 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 21:53:42.723439 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.714189    2577 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 21:53:42.723439 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.714192    2577 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 21:53:42.723439 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.714195    2577 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 21:53:42.723439 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.714197    2577 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 21:53:42.723439 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.714200    2577 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 21:53:42.723439 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.714203    2577 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 21:53:42.723439 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.714205    2577 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 21:53:42.723439 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.714208    2577 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 21:53:42.723439 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.714210    2577 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 21:53:42.723439 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.714213    2577 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 21:53:42.723439 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.714217    2577 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 21:53:42.723439 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.714220    2577 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 21:53:42.723439 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.714222    2577 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 21:53:42.723439 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.714225    2577 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 21:53:42.723439 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.714228    2577 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 21:53:42.723439 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.714231    2577 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 21:53:42.723439 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.714234    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 21:53:42.723439 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.714236    2577 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 21:53:42.723911 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.714239    2577 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 21:53:42.723911 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.714242    2577 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 21:53:42.723911 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.714244    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 21:53:42.723911 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.714247    2577 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 21:53:42.723911 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.714249    2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 21:53:42.723911 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.714252    2577 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 21:53:42.723911 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.714255    2577 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 21:53:42.723911 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.714257    2577 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 21:53:42.723911 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.714260    2577 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 21:53:42.723911 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.714262    2577 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 21:53:42.723911 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.714265    2577 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 21:53:42.723911 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.714268    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 21:53:42.723911 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:42.714297    2577 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 30 21:53:42.723911 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.714336    2577 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 21:53:42.723911 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.714341    2577 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 21:53:42.724289 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.714345    2577 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 21:53:42.724289 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.714349    2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 21:53:42.724289 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.714351    2577 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 21:53:42.724289 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.714354    2577 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 21:53:42.724289 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.714357    2577 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 21:53:42.724289 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.714360    2577 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 21:53:42.724289 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.714362    2577 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 21:53:42.724289 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.714365    2577 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 21:53:42.724289 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.714368    2577 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 21:53:42.724289 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.714372    2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 21:53:42.724289 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.714375    2577 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 21:53:42.724289 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.714378    2577 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 21:53:42.724289 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.714380    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 21:53:42.724289 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.714383    2577 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 21:53:42.724289 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.714385    2577 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 21:53:42.724289 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.714388    2577 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 21:53:42.724289 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.714391    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 21:53:42.724289 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.714394    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 21:53:42.724289 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.714397    2577 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 21:53:42.724289 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.714399    2577 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 21:53:42.724756 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.714403    2577 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 21:53:42.724756 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.714407    2577 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 21:53:42.724756 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.714411    2577 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 21:53:42.724756 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.714413    2577 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 21:53:42.724756 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.714416    2577 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 21:53:42.724756 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.714419    2577 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 21:53:42.724756 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.714421    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 21:53:42.724756 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.714424    2577 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 21:53:42.724756 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.714427    2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 21:53:42.724756 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.714429    2577 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 21:53:42.724756 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.714431    2577 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 21:53:42.724756 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.714435    2577 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 21:53:42.724756 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.714438    2577 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 21:53:42.724756 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.714442    2577 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 21:53:42.724756 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.714447    2577 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 21:53:42.724756 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.714450    2577 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 21:53:42.724756 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.714453    2577 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 21:53:42.724756 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.714456    2577 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 21:53:42.724756 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.714458    2577 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 21:53:42.724756 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.714461    2577 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 21:53:42.725254 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.714463    2577 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 21:53:42.725254 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.714468    2577 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 21:53:42.725254 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.714471    2577 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 21:53:42.725254 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.714473    2577 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 21:53:42.725254 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.714476    2577 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 21:53:42.725254 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.714479    2577 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 21:53:42.725254 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.714481    2577 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 21:53:42.725254 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.714484    2577 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 21:53:42.725254 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.714486    2577 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 21:53:42.725254 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.714489    2577 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 21:53:42.725254 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.714492    2577 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 21:53:42.725254 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.714495    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 21:53:42.725254 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.714498    2577 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 21:53:42.725254 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.714500    2577 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 21:53:42.725254 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.714503    2577 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 21:53:42.725254 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.714505    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 21:53:42.725254 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.714508    2577 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 21:53:42.725254 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.714510    2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 21:53:42.725254 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.714513    2577 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 21:53:42.725714 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.714515    2577 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 21:53:42.725714 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.714518    2577 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 21:53:42.725714 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.714520    2577 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 21:53:42.725714 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.714523    2577 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 21:53:42.725714 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.714527    2577 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 21:53:42.725714 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.714530    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 21:53:42.725714 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.714532    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 21:53:42.725714 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.714535    2577 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 21:53:42.725714 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.714538    2577 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 21:53:42.725714 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.714541    2577 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 21:53:42.725714 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.714543    2577 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 21:53:42.725714 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.714546    2577 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 21:53:42.725714 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.714549    2577 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 21:53:42.725714 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.714551    2577 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 21:53:42.725714 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.714555    2577 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 21:53:42.725714 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.714557    2577 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 21:53:42.725714 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.714560    2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 21:53:42.725714 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.714563    2577 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 21:53:42.725714 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.714565    2577 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 21:53:42.726168 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.714568    2577 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 21:53:42.726168 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.714571    2577 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 21:53:42.726168 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.714574    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 21:53:42.726168 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.714576    2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 21:53:42.726168 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.714579    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 21:53:42.726168 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.714582    2577 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 21:53:42.726168 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.714584    2577 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 21:53:42.726168 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.714587    2577 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 21:53:42.726168 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.714589    2577 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 21:53:42.726168 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.714592    2577 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 21:53:42.726168 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.714594    2577 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 21:53:42.726168 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.714596    2577 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 21:53:42.726168 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.714599    2577 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 21:53:42.726168 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.714602    2577 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 21:53:42.726168 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.714604    2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 21:53:42.726168 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.714606    2577 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 21:53:42.726168 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.714609    2577 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 21:53:42.726168 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.714614    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 21:53:42.726168 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.714616    2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 21:53:42.726647 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.714619    2577 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 21:53:42.726647 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.714622    2577 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 21:53:42.726647 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.714626    2577 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 21:53:42.726647 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.714628    2577 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 21:53:42.726647 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.714631    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 21:53:42.726647 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.714633    2577 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 21:53:42.726647 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.714636    2577 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 21:53:42.726647 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:42.721530    2577 server.go:525] "Kubelet version" kubeletVersion="v1.35.9"
Oct 30 21:53:42.726647 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:42.721550    2577 server.go:527] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Oct 30 21:53:42.726647 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:42.722193    2577 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Oct 30 21:53:42.726647 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:42.722199    2577 watchdog_linux.go:138] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Oct 30 21:53:42.726647 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.722273    2577 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 21:53:42.726647 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.722279    2577 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 21:53:42.726647 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.722283    2577 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 21:53:42.726647 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.722286    2577 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 21:53:42.726647 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.722289    2577 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 21:53:42.726647 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.722292    2577 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 21:53:42.726647 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.722295    2577 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 21:53:42.726647 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.722298    2577 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 21:53:42.727146 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.722301    2577 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 21:53:42.727146 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.722303    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 21:53:42.727146 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.722306    2577 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 21:53:42.727146 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.722308    2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 21:53:42.727146 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.722311    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 21:53:42.727146 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.722314    2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 21:53:42.727146 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.722317    2577 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 21:53:42.727146 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.722319    2577 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 21:53:42.727146 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.722322    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 21:53:42.727146 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.722325    2577 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 21:53:42.727146 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.722327    2577 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 21:53:42.727146 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.722330    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 21:53:42.727146 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.722332    2577 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 21:53:42.727146 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.722335    2577 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 21:53:42.727146 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.722338    2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 21:53:42.727146 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.722341    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 21:53:42.727146 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.722343    2577 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 21:53:42.727146 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.722346    2577 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 21:53:42.727146 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.722349    2577 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 21:53:42.727599 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.722351    2577 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 21:53:42.727599 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.722354    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 21:53:42.727599 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.722359    2577 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 21:53:42.727599 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.722361    2577 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 21:53:42.727599 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.722364    2577 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 21:53:42.727599 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.722367    2577 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 21:53:42.727599 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.722370    2577 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 21:53:42.727599 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.722372    2577 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 21:53:42.727599 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.722375    2577 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 21:53:42.727599 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.722378    2577 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 21:53:42.727599 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.722380    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 21:53:42.727599 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.722383    2577 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 21:53:42.727599 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.722385    2577 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 21:53:42.727599 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.722388    2577 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 21:53:42.727599 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.722391    2577 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 21:53:42.727599 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.722394    2577 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 21:53:42.727599 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.722397    2577 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 21:53:42.727599 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.722399    2577 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 21:53:42.727599 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.722402    2577 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 21:53:42.727599 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.722405    2577 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 21:53:42.728104 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.722408    2577 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 21:53:42.728104 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.722410    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 21:53:42.728104 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.722413    2577 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 21:53:42.728104 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.722415    2577 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 21:53:42.728104 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.722418    2577 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 21:53:42.728104 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.722421    2577 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 21:53:42.728104 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.722424    2577 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 21:53:42.728104 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.722426    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 21:53:42.728104 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.722429    2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 21:53:42.728104 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.722431    2577 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 21:53:42.728104 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.722434    2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 21:53:42.728104 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.722437    2577 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 21:53:42.728104 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.722440    2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 21:53:42.728104 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.722443    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 21:53:42.728104 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.722446    2577 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 21:53:42.728104 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.722450    2577 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 21:53:42.728104 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.722453    2577 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 21:53:42.728104 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.722455    2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 21:53:42.728104 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.722458    2577 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 21:53:42.728613 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.722462    2577 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 21:53:42.728613 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.722467    2577 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 21:53:42.728613 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.722470    2577 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 21:53:42.728613 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.722474    2577 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 21:53:42.728613 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.722477    2577 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 21:53:42.728613 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.722480    2577 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 21:53:42.728613 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.722482    2577 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 21:53:42.728613 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.722485    2577 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 21:53:42.728613 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.722488    2577 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 21:53:42.728613 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.722492    2577 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 21:53:42.728613 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.722495    2577 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 21:53:42.728613 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.722498    2577 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 21:53:42.728613 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.722501    2577 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 21:53:42.728613 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.722504    2577 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 21:53:42.728613 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.722506    2577 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 21:53:42.728613 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.722509    2577 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 21:53:42.728613 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.722512    2577 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 21:53:42.728613 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.722514    2577 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 21:53:42.728613 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.722517    2577 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 21:53:42.729089 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.722520    2577 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 21:53:42.729089 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.722522    2577 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 21:53:42.729089 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.722525    2577 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 21:53:42.729089 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.722527    2577 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 21:53:42.729089 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.722530    2577 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 21:53:42.729089 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.722532    2577 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 21:53:42.729089 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.722535    2577 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 21:53:42.729089 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.722538    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 21:53:42.729089 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.722541    2577 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 21:53:42.729089 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.722545    2577 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 21:53:42.729089 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.722548    2577 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 21:53:42.729089 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.722551    2577 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 21:53:42.729089 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.722554    2577 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 21:53:42.729089 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.722556    2577 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 21:53:42.729089 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.722559    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 21:53:42.729089 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.722561    2577 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 21:53:42.729089 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.722564    2577 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 21:53:42.729089 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.722566    2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 21:53:42.729089 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.722569    2577 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 21:53:42.729089 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.722572    2577 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 21:53:42.729599 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.722574    2577 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 21:53:42.729599 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:42.722606    2577 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 30 21:53:42.729599 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.722759    2577 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 21:53:42.729599 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.722766    2577 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 21:53:42.729599 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.722769    2577 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 21:53:42.729599 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.722772    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 21:53:42.729599 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.722775    2577 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 21:53:42.729599 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.722778    2577 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 21:53:42.729599 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.722780    2577 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 21:53:42.729599 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.722783    2577 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 21:53:42.729599 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.722786    2577 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 21:53:42.729599 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.722788    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 21:53:42.729599 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.722791    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 21:53:42.729599 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.722794    2577 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 21:53:42.729599 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.722796    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 21:53:42.729599 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.722799    2577 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 21:53:42.729974 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.722802    2577 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 21:53:42.729974 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.722804    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 21:53:42.729974 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.722807    2577 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 21:53:42.729974 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.722809    2577 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 21:53:42.729974 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.722813    2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 21:53:42.729974 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.722815    2577 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 21:53:42.729974 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.722818    2577 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 21:53:42.729974 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.722822    2577 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 21:53:42.729974 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.722825    2577 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 21:53:42.729974 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.722827    2577 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 21:53:42.729974 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.722830    2577 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 21:53:42.729974 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.722833    2577 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 21:53:42.729974 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.722835    2577 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 21:53:42.729974 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.722838    2577 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 21:53:42.729974 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.722841    2577 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 21:53:42.729974 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.722843    2577 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 21:53:42.729974 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.722846    2577 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 21:53:42.729974 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.722849    2577 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 21:53:42.729974 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.722851    2577 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 21:53:42.729974 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.722854    2577 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 21:53:42.730511 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.722856    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 21:53:42.730511 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.722859    2577 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 21:53:42.730511 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.722861    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 21:53:42.730511 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.722864    2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 21:53:42.730511 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.722866    2577 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 21:53:42.730511 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.722869    2577 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 21:53:42.730511 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.722872    2577 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 21:53:42.730511 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.722874    2577 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 21:53:42.730511 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.722877    2577 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 21:53:42.730511 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.722879    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 21:53:42.730511 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.722882    2577 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 21:53:42.730511 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.722884    2577 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 21:53:42.730511 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.722887    2577 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 21:53:42.730511 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.722889    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 21:53:42.730511 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.722892    2577 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 21:53:42.730511 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.722895    2577 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 21:53:42.730511 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.722898    2577 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 21:53:42.730511 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.722901    2577 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 21:53:42.730511 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.722905    2577 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 21:53:42.730511 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.722907    2577 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 21:53:42.731022 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.722911    2577 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 21:53:42.731022 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.722913    2577 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 21:53:42.731022 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.722916    2577 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 21:53:42.731022 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.722918    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 21:53:42.731022 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.722921    2577 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 21:53:42.731022 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.722923    2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 21:53:42.731022 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.722926    2577 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 21:53:42.731022 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.722928    2577 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 21:53:42.731022 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.722931    2577 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 21:53:42.731022 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.722933    2577 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 21:53:42.731022 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.722936    2577 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 21:53:42.731022 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.722938    2577 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 21:53:42.731022 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.722941    2577 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 21:53:42.731022 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.722943    2577 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 21:53:42.731022 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.722946    2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 21:53:42.731022 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.722948    2577 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 21:53:42.731022 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.722951    2577 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 21:53:42.731022 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.722954    2577 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 21:53:42.731022 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.722956    2577 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 21:53:42.731022 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.722959    2577 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 21:53:42.731602 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.722962    2577 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 21:53:42.731602 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.722964    2577 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 21:53:42.731602 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.722967    2577 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 21:53:42.731602 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.722970    2577 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 21:53:42.731602 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.722973    2577 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 21:53:42.731602 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.722975    2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 21:53:42.731602 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.722978    2577 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 21:53:42.731602 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.722980    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 21:53:42.731602 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.722984    2577 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 21:53:42.731602 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.723002    2577 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 21:53:42.731602 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.723007    2577 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 21:53:42.731602 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.723010    2577 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 21:53:42.731602 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.723013    2577 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 21:53:42.731602 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.723016    2577 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 21:53:42.731602 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.723019    2577 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 21:53:42.731602 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.723022    2577 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 21:53:42.731602 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.723024    2577 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 21:53:42.731602 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.723033    2577 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 21:53:42.731602 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.723037    2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 21:53:42.731602 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.723039    2577 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 21:53:42.732499 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.723042    2577 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 21:53:42.732499 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.723045    2577 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 21:53:42.732499 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.723048    2577 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 21:53:42.732499 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.723050    2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 21:53:42.732499 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.723053    2577 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 21:53:42.732499 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.723055    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 21:53:42.732499 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.723058    2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 21:53:42.732499 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.723061    2577 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 21:53:42.732499 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.723064    2577 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 21:53:42.732499 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.723067    2577 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 21:53:42.732499 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.723071    2577 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 21:53:42.732499 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.723075    2577 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 21:53:42.732499 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:42.723102    2577 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 30 21:53:42.732499 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.723136    2577 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 21:53:42.732499 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.723141    2577 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 21:53:42.732966 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.723145    2577 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 21:53:42.732966 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.723147    2577 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 21:53:42.732966 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.723150    2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 21:53:42.732966 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.723152    2577 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 21:53:42.732966 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.723155    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 21:53:42.732966 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.723159    2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 21:53:42.732966 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.723161    2577 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 21:53:42.732966 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.723164    2577 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 21:53:42.732966 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.723168    2577 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 21:53:42.732966 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.723172    2577 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 21:53:42.732966 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.723176    2577 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 21:53:42.732966 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.723181    2577 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 21:53:42.732966 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.723185    2577 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 21:53:42.732966 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.723188    2577 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 21:53:42.732966 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.723190    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 21:53:42.732966 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.723193    2577 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 21:53:42.732966 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.723195    2577 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 21:53:42.732966 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.723198    2577 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 21:53:42.732966 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.723201    2577 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 21:53:42.733522 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.723203    2577 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 21:53:42.733522 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.723206    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 21:53:42.733522 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.723209    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 21:53:42.733522 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.723211    2577 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 21:53:42.733522 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.723214    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 21:53:42.733522 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.723216    2577 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 21:53:42.733522 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.723219    2577 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 21:53:42.733522 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.723222    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 21:53:42.733522 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.723225    2577 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 21:53:42.733522 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.723227    2577 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 21:53:42.733522 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.723230    2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 21:53:42.733522 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.723233    2577 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 21:53:42.733522 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.723235    2577 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 21:53:42.733522 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.723238    2577 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 21:53:42.733522 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.723240    2577 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 21:53:42.733522 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.723243    2577 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 21:53:42.733522 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.723245    2577 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 21:53:42.733522 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.723249    2577 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 21:53:42.733522 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.723252    2577 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 21:53:42.734023 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.723254    2577 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 21:53:42.734023 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.723257    2577 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 21:53:42.734023 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.723260    2577 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 21:53:42.734023 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.723262    2577 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 21:53:42.734023 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.723265    2577 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 21:53:42.734023 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.723267    2577 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 21:53:42.734023 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.723270    2577 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 21:53:42.734023 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.723273    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 21:53:42.734023 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.723275    2577 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 21:53:42.734023 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.723278    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 21:53:42.734023 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.723281    2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 21:53:42.734023 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.723283    2577 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 21:53:42.734023 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.723286    2577 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 21:53:42.734023 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.723289    2577 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 21:53:42.734023 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.723291    2577 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 21:53:42.734023 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.723294    2577 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 21:53:42.734023 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.723299    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 21:53:42.734023 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.723302    2577 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 21:53:42.734023 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.723304    2577 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 21:53:42.734023 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.723307    2577 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 21:53:42.734680 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.723310    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 21:53:42.734680 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.723313    2577 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 21:53:42.734680 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.723316    2577 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 21:53:42.734680 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.723319    2577 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 21:53:42.734680 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.723321    2577 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 21:53:42.734680 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.723324    2577 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 21:53:42.734680 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.723326    2577 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 21:53:42.734680 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.723329    2577 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 21:53:42.734680 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.723332    2577 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 21:53:42.734680 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.723334    2577 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 21:53:42.734680 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.723337    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 21:53:42.734680 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.723339    2577 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 21:53:42.734680 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.723342    2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 21:53:42.734680 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.723344    2577 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 21:53:42.734680 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.723347    2577 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 21:53:42.734680 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.723350    2577 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 21:53:42.734680 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.723352    2577 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 21:53:42.734680 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.723355    2577 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 21:53:42.734680 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.723357    2577 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 21:53:42.734680 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.723360    2577 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 21:53:42.735188 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.723363    2577 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 21:53:42.735188 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.723366    2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 21:53:42.735188 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.723368    2577 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 21:53:42.735188 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.723371    2577 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 21:53:42.735188 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.723374    2577 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 21:53:42.735188 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.723376    2577 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 21:53:42.735188 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.723379    2577 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 21:53:42.735188 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.723382    2577 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 21:53:42.735188 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.723386    2577 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 21:53:42.735188 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.723388    2577 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 21:53:42.735188 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.723391    2577 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 21:53:42.735188 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.723394    2577 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 21:53:42.735188 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.723396    2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 21:53:42.735188 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.723399    2577 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 21:53:42.735188 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.723402    2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 21:53:42.735188 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.723404    2577 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 21:53:42.735188 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.723407    2577 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 21:53:42.735188 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.723410    2577 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 21:53:42.735188 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.723412    2577 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 21:53:42.735188 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.723416    2577 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 21:53:42.735649 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.723418    2577 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 21:53:42.735649 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.723421    2577 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 21:53:42.735649 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.723423    2577 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 21:53:42.735649 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.723426    2577 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 21:53:42.735649 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.723428    2577 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 21:53:42.735649 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:42.723431    2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 21:53:42.735649 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:42.723505    2577 server.go:957] "Client rotation is on, will bootstrap in background"
Oct 30 21:53:42.735649 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:42.726647    2577 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Oct 30 21:53:42.735649 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:42.727630    2577 server.go:1014] "Starting client certificate rotation"
Oct 30 21:53:42.735649 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:42.727730    2577 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Oct 30 21:53:42.735649 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:42.727765    2577 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Oct 30 21:53:42.735649 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:42.730154    2577 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Oct 30 21:53:42.735649 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:42.732557    2577 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Oct 30 21:53:42.742513 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:42.742492    2577 log.go:25] "Validated CRI v1 runtime API"
Oct 30 21:53:42.745556 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:42.745538    2577 log.go:25] "Validated CRI v1 image API"
Oct 30 21:53:42.746361 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:42.746334    2577 server.go:1424] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Oct 30 21:53:42.749640 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:42.749593    2577 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 e7f760cd-c531-4205-a373-dccc3d16141d:/dev/nvme0n1p3 ede47d30-7089-4072-bf4e-c4f450f09bf8:/dev/nvme0n1p4]
Oct 30 21:53:42.749640 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:42.749613    2577 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:24 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:26 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:32 fsType:tmpfs blockSize:0} composefs_0-34:{mountpoint:/ major:0 minor:34 fsType:overlay blockSize:0}]
Oct 30 21:53:42.755012 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:42.754878    2577 manager.go:217] Machine: {Timestamp:2026-10-30 21:53:42.753535439 +0000 UTC m=+0.315948273 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3100854 MemoryCapacity:33164337152 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2cd0a68206bbc02ac459ae78208eb8 SystemUUID:ec2cd0a6-8206-bbc0-2ac4-59ae78208eb8 BootID:aa5b2c7a-2c32-463a-a8cc-603cc9c8ac89 Filesystems:[{Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:32 Capacity:16582168576 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:24 Capacity:16582168576 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:26 Capacity:6632869888 Type:vfs Inodes:819200 HasInodes:true} {Device:composefs_0-34 DeviceMajor:0 DeviceMinor:34 Capacity:6422528 Type:vfs Inodes:18446744073709551615 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:ff:ca:14:98:c9 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:ff:ca:14:98:c9 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:5a:bd:df:2b:0b:ea Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164337152 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Oct 30 21:53:42.755012 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:42.755010    2577 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Oct 30 21:53:42.755125 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:42.755110    2577 manager.go:233] Version: {KernelVersion:5.14.0-687.60.1.el9_8.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.8.20261023-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Oct 30 21:53:42.757877 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:42.757848    2577 container_manager_linux.go:275] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Oct 30 21:53:42.758086 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:42.757882    2577 container_manager_linux.go:280] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-143-246.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"/system.slice","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{},"system-reserved-compressible":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Oct 30 21:53:42.758129 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:42.758089    2577 topology_manager.go:143] "Creating topology manager with none policy"
Oct 30 21:53:42.758129 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:42.758098    2577 container_manager_linux.go:311] "Creating device plugin manager"
Oct 30 21:53:42.758129 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:42.758125    2577 manager.go:144] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Oct 30 21:53:42.758820 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:42.758810    2577 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Oct 30 21:53:42.759554 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:42.759543    2577 container_manager_linux.go:320] "Creating Dynamic Resource Allocation (DRA) manager"
Oct 30 21:53:42.819432 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:42.819396    2577 reflector.go:446] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Oct 30 21:53:42.823454 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:42.823429    2577 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-g2xlm"
Oct 30 21:53:42.830831 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:42.830805    2577 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-g2xlm"
Oct 30 21:53:42.903148 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:42.903123    2577 state_mem.go:41] "Initialized" logger="CPUManager state memory"
Oct 30 21:53:42.903295 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:42.903285    2577 server.go:1235] "Using root directory" path="/var/lib/kubelet"
Oct 30 21:53:42.903353 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:42.903322    2577 kubelet.go:493] "Attempting to sync node with API server"
Oct 30 21:53:42.903353 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:42.903343    2577 kubelet.go:394] "Adding static pod path" path="/etc/kubernetes/manifests"
Oct 30 21:53:42.903439 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:42.903360    2577 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Oct 30 21:53:42.903439 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:42.903371    2577 kubelet.go:405] "Adding apiserver pod source"
Oct 30 21:53:42.903439 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:42.903379    2577 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Oct 30 21:53:42.903777 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:42.903762    2577 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Oct 30 21:53:42.905663 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:42.905640    2577 kuberuntime_manager.go:304] "Container runtime initialized" containerRuntime="cri-o" version="1.35.10-3.rhaos4.22.git9383502.el9" apiVersion="v1"
Oct 30 21:53:42.912573 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:42.912549    2577 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Oct 30 21:53:42.913438 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:42.913424    2577 kubelet.go:963] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Oct 30 21:53:42.913503 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:42.913455    2577 kubelet.go:990] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Oct 30 21:53:42.913623 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:42.913613    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Oct 30 21:53:42.913677 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:42.913629    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Oct 30 21:53:42.913677 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:42.913636    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Oct 30 21:53:42.913677 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:42.913642    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Oct 30 21:53:42.913677 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:42.913648    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Oct 30 21:53:42.913677 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:42.913654    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Oct 30 21:53:42.914179 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:42.914171    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Oct 30 21:53:42.914179 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:42.914179    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Oct 30 21:53:42.914296 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:42.914187    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Oct 30 21:53:42.914296 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:42.914194    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Oct 30 21:53:42.914296 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:42.914203    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Oct 30 21:53:42.914296 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:42.914212    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Oct 30 21:53:42.915536 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:42.915524    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Oct 30 21:53:42.915536 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:42.915536    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Oct 30 21:53:42.919434 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:42.919414    2577 server.go:1263] "Started kubelet"
Oct 30 21:53:42.919575 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:42.919513    2577 server.go:182] "Starting to listen" address="0.0.0.0" port=10250
Oct 30 21:53:42.919677 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:42.919622    2577 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Oct 30 21:53:42.919711 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:42.919705    2577 server_v1.go:49] "podresources" method="list" useActivePods=true
Oct 30 21:53:42.920296 ip-10-0-143-246 systemd[1]: Started Kubernetes Kubelet.
Oct 30 21:53:42.921523 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:42.921364    2577 server.go:317] "Adding debug handlers to kubelet server"
Oct 30 21:53:42.922677 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:42.922659    2577 server.go:254] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Oct 30 21:53:42.925598 ip-10-0-143-246 kubenswrapper[2577]: E1030 21:53:42.925575    2577 kubelet.go:1679] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Oct 30 21:53:42.926771 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:42.926755    2577 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Oct 30 21:53:42.927238 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:42.927217    2577 fs_resource_analyzer.go:69] "Starting FS ResourceAnalyzer"
Oct 30 21:53:42.927459 ip-10-0-143-246 kubenswrapper[2577]: E1030 21:53:42.927429    2577 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-143-246.ec2.internal\" not found"
Oct 30 21:53:42.928088 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:42.928067    2577 factory.go:55] Registering systemd factory
Oct 30 21:53:42.928088 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:42.928089    2577 factory.go:223] Registration of the systemd container factory successfully
Oct 30 21:53:42.928334 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:42.928320    2577 factory.go:153] Registering CRI-O factory
Oct 30 21:53:42.928334 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:42.928336    2577 factory.go:223] Registration of the crio container factory successfully
Oct 30 21:53:42.928434 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:42.928380    2577 volume_manager.go:309] "The desired_state_of_world populator starts"
Oct 30 21:53:42.928434 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:42.928389    2577 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Oct 30 21:53:42.928434 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:42.928405    2577 factory.go:103] Registering Raw factory
Oct 30 21:53:42.928434 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:42.928380    2577 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Oct 30 21:53:42.928434 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:42.928396    2577 volume_manager.go:311] "Starting Kubelet Volume Manager"
Oct 30 21:53:42.928603 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:42.928420    2577 manager.go:1199] Started watching for new ooms in manager
Oct 30 21:53:42.928603 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:42.928586    2577 reconstruct.go:97] "Volume reconstruction finished"
Oct 30 21:53:42.928603 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:42.928596    2577 reconciler.go:29] "Reconciler: start to sync state"
Oct 30 21:53:42.929423 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:42.929404    2577 manager.go:319] Starting recovery of all containers
Oct 30 21:53:42.941613 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:42.941587    2577 manager.go:324] Recovery completed
Oct 30 21:53:42.943138 ip-10-0-143-246 kubenswrapper[2577]: E1030 21:53:42.943102    2577 watcher.go:152] Failed to watch directory "/sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service": inotify_add_watch /sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service: no such file or directory
Oct 30 21:53:42.946622 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:42.946602    2577 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 21:53:42.948751 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:42.948733    2577 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-143-246.ec2.internal" event="NodeHasSufficientMemory"
Oct 30 21:53:42.948814 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:42.948767    2577 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-143-246.ec2.internal" event="NodeHasNoDiskPressure"
Oct 30 21:53:42.948814 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:42.948778    2577 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-143-246.ec2.internal" event="NodeHasSufficientPID"
Oct 30 21:53:42.949231 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:42.949214    2577 reflector.go:446] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 21:53:42.949487 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:42.949475    2577 cpu_manager.go:226] "Starting" policy="none"
Oct 30 21:53:42.949487 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:42.949486    2577 cpu_manager.go:227] "Reconciling" reconcilePeriod="10s"
Oct 30 21:53:42.949557 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:42.949507    2577 state_mem.go:41] "Initialized" logger="CPUManager state checkpoint.CPUManager state memory"
Oct 30 21:53:42.949557 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:42.949480    2577 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 21:53:42.950931 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:42.950917    2577 reflector.go:446] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 21:53:42.951594 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:42.951577    2577 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-143-246.ec2.internal" event="NodeHasSufficientMemory"
Oct 30 21:53:42.951677 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:42.951611    2577 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-143-246.ec2.internal" event="NodeHasNoDiskPressure"
Oct 30 21:53:42.951677 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:42.951630    2577 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-143-246.ec2.internal" event="NodeHasSufficientPID"
Oct 30 21:53:42.951892 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:42.951879    2577 policy_none.go:50] "Start"
Oct 30 21:53:42.951934 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:42.951896    2577 memory_manager.go:187] "Starting memorymanager" policy="None"
Oct 30 21:53:42.951934 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:42.951910    2577 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Oct 30 21:53:42.954067 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:42.954054    2577 policy_none.go:44] "Start"
Oct 30 21:53:42.960031 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:42.960012    2577 reflector.go:446] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 21:53:42.960135 ip-10-0-143-246 kubenswrapper[2577]: E1030 21:53:42.960031    2577 nodelease.go:50] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-143-246.ec2.internal\" not found" node="ip-10-0-143-246.ec2.internal"
Oct 30 21:53:42.960135 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:42.960015    2577 nodeinfomanager.go:422] Failed to publish CSINode: nodes "ip-10-0-143-246.ec2.internal" not found
Oct 30 21:53:42.983597 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:42.983581    2577 nodeinfomanager.go:422] Failed to publish CSINode: nodes "ip-10-0-143-246.ec2.internal" not found
Oct 30 21:53:42.991705 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:42.991670    2577 node_container_manager_linux.go:130] "Enforcing system reserved compressible on cgroup" cgroupName="/system.slice" limits={"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"}
Oct 30 21:53:42.994476 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:42.994458    2577 manager.go:341] "Starting Device Plugin manager"
Oct 30 21:53:42.994574 ip-10-0-143-246 kubenswrapper[2577]: E1030 21:53:42.994489    2577 manager.go:525] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Oct 30 21:53:42.994574 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:42.994499    2577 server.go:87] "Starting device plugin registration server"
Oct 30 21:53:42.994793 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:42.994778    2577 eviction_manager.go:194] "Eviction manager: starting control loop"
Oct 30 21:53:42.994855 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:42.994793    2577 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Oct 30 21:53:42.994916 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:42.994891    2577 plugin_watcher.go:53] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Oct 30 21:53:42.995021 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:42.994983    2577 plugin_manager.go:119] "The desired_state_of_world populator (plugin watcher) starts"
Oct 30 21:53:42.995021 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:42.995011    2577 plugin_manager.go:121] "Starting Kubelet Plugin Manager"
Oct 30 21:53:42.995618 ip-10-0-143-246 kubenswrapper[2577]: E1030 21:53:42.995600    2577 eviction_manager.go:272] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Oct 30 21:53:42.995710 ip-10-0-143-246 kubenswrapper[2577]: E1030 21:53:42.995647    2577 eviction_manager.go:297] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-143-246.ec2.internal\" not found"
Oct 30 21:53:43.047665 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:43.047641    2577 nodeinfomanager.go:422] Failed to publish CSINode: nodes "ip-10-0-143-246.ec2.internal" not found
Oct 30 21:53:43.062493 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:43.062458    2577 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Oct 30 21:53:43.063827 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:43.063804    2577 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Oct 30 21:53:43.063951 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:43.063834    2577 status_manager.go:249] "Starting to sync pod status with apiserver"
Oct 30 21:53:43.063951 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:43.063859    2577 kubelet.go:2521] "Starting kubelet main sync loop"
Oct 30 21:53:43.063951 ip-10-0-143-246 kubenswrapper[2577]: E1030 21:53:43.063900    2577 kubelet.go:2545] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Oct 30 21:53:43.077425 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:43.077394    2577 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 21:53:43.095859 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:43.095822    2577 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 21:53:43.096809 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:43.096791    2577 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-143-246.ec2.internal" event="NodeHasSufficientMemory"
Oct 30 21:53:43.096910 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:43.096821    2577 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-143-246.ec2.internal" event="NodeHasNoDiskPressure"
Oct 30 21:53:43.096910 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:43.096831    2577 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-143-246.ec2.internal" event="NodeHasSufficientPID"
Oct 30 21:53:43.096910 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:43.096855    2577 kubelet_node_status.go:77] "Attempting to register node" node="ip-10-0-143-246.ec2.internal"
Oct 30 21:53:43.115406 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:43.115382    2577 kubelet_node_status.go:80] "Successfully registered node" node="ip-10-0-143-246.ec2.internal"
Oct 30 21:53:43.164634 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:43.164599    2577 kubelet.go:2608] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-246.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-143-246.ec2.internal"]
Oct 30 21:53:43.167098 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:43.167076    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-246.ec2.internal"
Oct 30 21:53:43.168504 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:43.168488    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-246.ec2.internal"
Oct 30 21:53:43.203087 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:43.202829    2577 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-246.ec2.internal"
Oct 30 21:53:43.205558 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:43.205540    2577 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-246.ec2.internal"
Oct 30 21:53:43.230478 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:43.230394    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/b7eadc83c6b2f891221807815b7cecbc-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-246.ec2.internal\" (UID: \"b7eadc83c6b2f891221807815b7cecbc\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-246.ec2.internal"
Oct 30 21:53:43.230478 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:43.230427    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b7eadc83c6b2f891221807815b7cecbc-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-246.ec2.internal\" (UID: \"b7eadc83c6b2f891221807815b7cecbc\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-246.ec2.internal"
Oct 30 21:53:43.230478 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:43.230448    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/b29468cdb315509fa1bb87ee196f0663-config\") pod \"kube-apiserver-proxy-ip-10-0-143-246.ec2.internal\" (UID: \"b29468cdb315509fa1bb87ee196f0663\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-246.ec2.internal"
Oct 30 21:53:43.237713 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:43.237687    2577 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 30 21:53:43.237713 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:43.237712    2577 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 30 21:53:43.330850 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:43.330821    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/b7eadc83c6b2f891221807815b7cecbc-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-246.ec2.internal\" (UID: \"b7eadc83c6b2f891221807815b7cecbc\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-246.ec2.internal"
Oct 30 21:53:43.330850 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:43.330838    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/b7eadc83c6b2f891221807815b7cecbc-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-246.ec2.internal\" (UID: \"b7eadc83c6b2f891221807815b7cecbc\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-246.ec2.internal"
Oct 30 21:53:43.331021 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:43.330898    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b7eadc83c6b2f891221807815b7cecbc-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-246.ec2.internal\" (UID: \"b7eadc83c6b2f891221807815b7cecbc\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-246.ec2.internal"
Oct 30 21:53:43.331021 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:43.330921    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/b29468cdb315509fa1bb87ee196f0663-config\") pod \"kube-apiserver-proxy-ip-10-0-143-246.ec2.internal\" (UID: \"b29468cdb315509fa1bb87ee196f0663\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-246.ec2.internal"
Oct 30 21:53:43.331021 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:43.330968    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/b29468cdb315509fa1bb87ee196f0663-config\") pod \"kube-apiserver-proxy-ip-10-0-143-246.ec2.internal\" (UID: \"b29468cdb315509fa1bb87ee196f0663\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-246.ec2.internal"
Oct 30 21:53:43.331021 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:43.330983    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b7eadc83c6b2f891221807815b7cecbc-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-246.ec2.internal\" (UID: \"b7eadc83c6b2f891221807815b7cecbc\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-246.ec2.internal"
Oct 30 21:53:43.542086 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:43.542002    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-246.ec2.internal"
Oct 30 21:53:43.542241 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:43.542088    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-246.ec2.internal"
Oct 30 21:53:43.733286 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:43.733249    2577 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Oct 30 21:53:43.733931 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:43.733414    2577 reflector.go:578] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.Service" err="very short watch: k8s.io/client-go/informers/factory.go:161: Unexpected watch close - watch lasted less than a second and no items received"
Oct 30 21:53:43.733931 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:43.733412    2577 reflector.go:578] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:161: Unexpected watch close - watch lasted less than a second and no items received"
Oct 30 21:53:43.733931 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:43.733416    2577 reflector.go:578] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:161: Unexpected watch close - watch lasted less than a second and no items received"
Oct 30 21:53:43.904777 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:43.904676    2577 apiserver.go:52] "Watching apiserver"
Oct 30 21:53:43.904777 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:43.904684    2577 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-10-29 21:48:42 +0000 UTC" deadline="2028-07-09 00:32:43.728333386 +0000 UTC"
Oct 30 21:53:43.904777 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:43.904759    2577 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14810h38m59.823578802s"
Oct 30 21:53:43.927018 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:43.926973    2577 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Oct 30 21:53:43.953248 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:43.953219    2577 reflector.go:446] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Oct 30 21:53:43.953541 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:43.953523    2577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-network-operator/iptables-alerter-kghfh","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kqbp9","openshift-dns/node-resolver-wwb9f","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-246.ec2.internal","openshift-multus/network-metrics-daemon-n8xrh","openshift-ovn-kubernetes/ovnkube-node-mn9vl","kube-system/global-pull-secret-syncer-q7fwz","kube-system/kube-apiserver-proxy-ip-10-0-143-246.ec2.internal","openshift-cluster-node-tuning-operator/tuned-h8csb","openshift-image-registry/node-ca-qzgkj","openshift-multus/multus-additional-cni-plugins-c9tnl","openshift-multus/multus-jkdxt","kube-system/konnectivity-agent-thksb","openshift-network-diagnostics/network-check-target-fvbz8"]
Oct 30 21:53:43.956262 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:43.956245    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-kghfh"
Oct 30 21:53:43.958823 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:43.958797    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kqbp9"
Oct 30 21:53:43.961456 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:43.961434    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-wwb9f"
Oct 30 21:53:43.964520 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:43.964496    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-n8xrh"
Oct 30 21:53:43.964626 ip-10-0-143-246 kubenswrapper[2577]: E1030 21:53:43.964581    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-n8xrh" podUID="532658b6-c86e-4feb-89cb-14a91316a7a0"
Oct 30 21:53:43.966982 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:43.966950    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-mn9vl"
Oct 30 21:53:43.969894 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:43.969875    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-q7fwz"
Oct 30 21:53:43.969999 ip-10-0-143-246 kubenswrapper[2577]: E1030 21:53:43.969943    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-q7fwz" podUID="af83b188-09cb-4621-a9da-be33856d19f0"
Oct 30 21:53:43.972861 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:43.972844    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-h8csb"
Oct 30 21:53:43.974440 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:43.974416    2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-nhqdj\""
Oct 30 21:53:43.974789 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:43.974766    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Oct 30 21:53:43.974789 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:43.974781    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Oct 30 21:53:43.974789 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:43.974785    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Oct 30 21:53:43.975613 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:43.975423    2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Oct 30 21:53:43.975843 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:43.975826    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-qzgkj"
Oct 30 21:53:43.978767 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:43.978748    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-c9tnl"
Oct 30 21:53:43.982077 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:43.982056    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-jkdxt"
Oct 30 21:53:43.982724 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:43.982707    2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-28tsh\""
Oct 30 21:53:43.982962 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:43.982947    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Oct 30 21:53:43.983734 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:43.983718    2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-htkkz\""
Oct 30 21:53:43.983954 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:43.983940    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Oct 30 21:53:43.984328 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:43.984312    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Oct 30 21:53:43.984396 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:43.984382    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Oct 30 21:53:43.985418 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:43.985401    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-thksb"
Oct 30 21:53:43.986244 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:43.986230    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Oct 30 21:53:43.987776 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:43.987759    2577 reflector.go:446] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Oct 30 21:53:43.988469 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:43.988452    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fvbz8"
Oct 30 21:53:43.988552 ip-10-0-143-246 kubenswrapper[2577]: E1030 21:53:43.988502    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fvbz8" podUID="32ae0d42-4b76-47b7-b9d6-26b09f00ba5d"
Oct 30 21:53:44.000155 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.000134    2577 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-sn4kj"
Oct 30 21:53:44.010537 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.010514    2577 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-sn4kj"
Oct 30 21:53:44.030476 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.030449    2577 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Oct 30 21:53:44.036704 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.036675    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/af83b188-09cb-4621-a9da-be33856d19f0-kubelet-config\") pod \"global-pull-secret-syncer-q7fwz\" (UID: \"af83b188-09cb-4621-a9da-be33856d19f0\") " pod="kube-system/global-pull-secret-syncer-q7fwz"
Oct 30 21:53:44.036704 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.036705    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/af83b188-09cb-4621-a9da-be33856d19f0-original-pull-secret\") pod \"global-pull-secret-syncer-q7fwz\" (UID: \"af83b188-09cb-4621-a9da-be33856d19f0\") " pod="kube-system/global-pull-secret-syncer-q7fwz"
Oct 30 21:53:44.036904 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.036725    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r4qz4\" (UniqueName: \"kubernetes.io/projected/bebdb584-1a96-453d-8a6d-b332e47bec3d-kube-api-access-r4qz4\") pod \"tuned-h8csb\" (UID: \"bebdb584-1a96-453d-8a6d-b332e47bec3d\") " pod="openshift-cluster-node-tuning-operator/tuned-h8csb"
Oct 30 21:53:44.036904 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.036742    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/15e1de31-92a1-4179-8ec1-400346880d1d-device-dir\") pod \"aws-ebs-csi-driver-node-kqbp9\" (UID: \"15e1de31-92a1-4179-8ec1-400346880d1d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kqbp9"
Oct 30 21:53:44.036904 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.036814    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/446beacf-70a0-4d47-96ea-bbe1ca4a5f28-run-systemd\") pod \"ovnkube-node-mn9vl\" (UID: \"446beacf-70a0-4d47-96ea-bbe1ca4a5f28\") " pod="openshift-ovn-kubernetes/ovnkube-node-mn9vl"
Oct 30 21:53:44.036904 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.036853    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/446beacf-70a0-4d47-96ea-bbe1ca4a5f28-log-socket\") pod \"ovnkube-node-mn9vl\" (UID: \"446beacf-70a0-4d47-96ea-bbe1ca4a5f28\") " pod="openshift-ovn-kubernetes/ovnkube-node-mn9vl"
Oct 30 21:53:44.037149 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.036903    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5d5057fc-dd8a-4dce-b8bb-70b2dc3b074c-cni-binary-copy\") pod \"multus-jkdxt\" (UID: \"5d5057fc-dd8a-4dce-b8bb-70b2dc3b074c\") " pod="openshift-multus/multus-jkdxt"
Oct 30 21:53:44.037149 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.036943    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/5d5057fc-dd8a-4dce-b8bb-70b2dc3b074c-hostroot\") pod \"multus-jkdxt\" (UID: \"5d5057fc-dd8a-4dce-b8bb-70b2dc3b074c\") " pod="openshift-multus/multus-jkdxt"
Oct 30 21:53:44.037149 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.036977    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/532658b6-c86e-4feb-89cb-14a91316a7a0-metrics-certs\") pod \"network-metrics-daemon-n8xrh\" (UID: \"532658b6-c86e-4feb-89cb-14a91316a7a0\") " pod="openshift-multus/network-metrics-daemon-n8xrh"
Oct 30 21:53:44.037149 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.037057    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/446beacf-70a0-4d47-96ea-bbe1ca4a5f28-systemd-units\") pod \"ovnkube-node-mn9vl\" (UID: \"446beacf-70a0-4d47-96ea-bbe1ca4a5f28\") " pod="openshift-ovn-kubernetes/ovnkube-node-mn9vl"
Oct 30 21:53:44.037149 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.037113    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/446beacf-70a0-4d47-96ea-bbe1ca4a5f28-ovnkube-config\") pod \"ovnkube-node-mn9vl\" (UID: \"446beacf-70a0-4d47-96ea-bbe1ca4a5f28\") " pod="openshift-ovn-kubernetes/ovnkube-node-mn9vl"
Oct 30 21:53:44.037149 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.037145    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/5d5057fc-dd8a-4dce-b8bb-70b2dc3b074c-host-var-lib-cni-multus\") pod \"multus-jkdxt\" (UID: \"5d5057fc-dd8a-4dce-b8bb-70b2dc3b074c\") " pod="openshift-multus/multus-jkdxt"
Oct 30 21:53:44.037450 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.037170    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5d5057fc-dd8a-4dce-b8bb-70b2dc3b074c-multus-conf-dir\") pod \"multus-jkdxt\" (UID: \"5d5057fc-dd8a-4dce-b8bb-70b2dc3b074c\") " pod="openshift-multus/multus-jkdxt"
Oct 30 21:53:44.037450 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.037222    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/bebdb584-1a96-453d-8a6d-b332e47bec3d-run\") pod \"tuned-h8csb\" (UID: \"bebdb584-1a96-453d-8a6d-b332e47bec3d\") " pod="openshift-cluster-node-tuning-operator/tuned-h8csb"
Oct 30 21:53:44.037450 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.037258    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/bebdb584-1a96-453d-8a6d-b332e47bec3d-sys\") pod \"tuned-h8csb\" (UID: \"bebdb584-1a96-453d-8a6d-b332e47bec3d\") " pod="openshift-cluster-node-tuning-operator/tuned-h8csb"
Oct 30 21:53:44.037450 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.037293    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/3e60fceb-8682-45cc-a472-f3d7017c3395-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-c9tnl\" (UID: \"3e60fceb-8682-45cc-a472-f3d7017c3395\") " pod="openshift-multus/multus-additional-cni-plugins-c9tnl"
Oct 30 21:53:44.037450 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.037340    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/446beacf-70a0-4d47-96ea-bbe1ca4a5f28-host-kubelet\") pod \"ovnkube-node-mn9vl\" (UID: \"446beacf-70a0-4d47-96ea-bbe1ca4a5f28\") " pod="openshift-ovn-kubernetes/ovnkube-node-mn9vl"
Oct 30 21:53:44.037450 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.037382    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/446beacf-70a0-4d47-96ea-bbe1ca4a5f28-etc-openvswitch\") pod \"ovnkube-node-mn9vl\" (UID: \"446beacf-70a0-4d47-96ea-bbe1ca4a5f28\") " pod="openshift-ovn-kubernetes/ovnkube-node-mn9vl"
Oct 30 21:53:44.037450 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.037440    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5d5057fc-dd8a-4dce-b8bb-70b2dc3b074c-etc-kubernetes\") pod \"multus-jkdxt\" (UID: \"5d5057fc-dd8a-4dce-b8bb-70b2dc3b074c\") " pod="openshift-multus/multus-jkdxt"
Oct 30 21:53:44.037827 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.037463    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/2bcee03a-9b40-4155-8d81-43dece439162-konnectivity-ca\") pod \"konnectivity-agent-thksb\" (UID: \"2bcee03a-9b40-4155-8d81-43dece439162\") " pod="kube-system/konnectivity-agent-thksb"
Oct 30 21:53:44.037827 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.037483    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/3e60fceb-8682-45cc-a472-f3d7017c3395-cni-binary-copy\") pod \"multus-additional-cni-plugins-c9tnl\" (UID: \"3e60fceb-8682-45cc-a472-f3d7017c3395\") " pod="openshift-multus/multus-additional-cni-plugins-c9tnl"
Oct 30 21:53:44.037827 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.037506    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jp25c\" (UniqueName: \"kubernetes.io/projected/fc4a67d9-8c7c-4f71-9a8a-2bba09329bc6-kube-api-access-jp25c\") pod \"node-resolver-wwb9f\" (UID: \"fc4a67d9-8c7c-4f71-9a8a-2bba09329bc6\") " pod="openshift-dns/node-resolver-wwb9f"
Oct 30 21:53:44.037827 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.037523    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/446beacf-70a0-4d47-96ea-bbe1ca4a5f28-ovnkube-script-lib\") pod \"ovnkube-node-mn9vl\" (UID: \"446beacf-70a0-4d47-96ea-bbe1ca4a5f28\") " pod="openshift-ovn-kubernetes/ovnkube-node-mn9vl"
Oct 30 21:53:44.037827 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.037539    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/fe472529-1294-46a1-a666-dc00d5d37eca-serviceca\") pod \"node-ca-qzgkj\" (UID: \"fe472529-1294-46a1-a666-dc00d5d37eca\") " pod="openshift-image-registry/node-ca-qzgkj"
Oct 30 21:53:44.037827 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.037565    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5d5057fc-dd8a-4dce-b8bb-70b2dc3b074c-host-var-lib-cni-bin\") pod \"multus-jkdxt\" (UID: \"5d5057fc-dd8a-4dce-b8bb-70b2dc3b074c\") " pod="openshift-multus/multus-jkdxt"
Oct 30 21:53:44.037827 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.037580    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/446beacf-70a0-4d47-96ea-bbe1ca4a5f28-run-openvswitch\") pod \"ovnkube-node-mn9vl\" (UID: \"446beacf-70a0-4d47-96ea-bbe1ca4a5f28\") " pod="openshift-ovn-kubernetes/ovnkube-node-mn9vl"
Oct 30 21:53:44.037827 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.037603    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kncjh\" (UniqueName: \"kubernetes.io/projected/5d5057fc-dd8a-4dce-b8bb-70b2dc3b074c-kube-api-access-kncjh\") pod \"multus-jkdxt\" (UID: \"5d5057fc-dd8a-4dce-b8bb-70b2dc3b074c\") " pod="openshift-multus/multus-jkdxt"
Oct 30 21:53:44.037827 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.037620    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/bebdb584-1a96-453d-8a6d-b332e47bec3d-etc-tuned\") pod \"tuned-h8csb\" (UID: \"bebdb584-1a96-453d-8a6d-b332e47bec3d\") " pod="openshift-cluster-node-tuning-operator/tuned-h8csb"
Oct 30 21:53:44.037827 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.037636    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/613f514c-6011-4936-8df5-a59b960cd3a6-host-slash\") pod \"iptables-alerter-kghfh\" (UID: \"613f514c-6011-4936-8df5-a59b960cd3a6\") " pod="openshift-network-operator/iptables-alerter-kghfh"
Oct 30 21:53:44.037827 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.037651    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/446beacf-70a0-4d47-96ea-bbe1ca4a5f28-var-lib-openvswitch\") pod \"ovnkube-node-mn9vl\" (UID: \"446beacf-70a0-4d47-96ea-bbe1ca4a5f28\") " pod="openshift-ovn-kubernetes/ovnkube-node-mn9vl"
Oct 30 21:53:44.037827 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.037671    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/5d5057fc-dd8a-4dce-b8bb-70b2dc3b074c-host-run-multus-certs\") pod \"multus-jkdxt\" (UID: \"5d5057fc-dd8a-4dce-b8bb-70b2dc3b074c\") " pod="openshift-multus/multus-jkdxt"
Oct 30 21:53:44.037827 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.037688    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6bsvp\" (UniqueName: \"kubernetes.io/projected/3e60fceb-8682-45cc-a472-f3d7017c3395-kube-api-access-6bsvp\") pod \"multus-additional-cni-plugins-c9tnl\" (UID: \"3e60fceb-8682-45cc-a472-f3d7017c3395\") " pod="openshift-multus/multus-additional-cni-plugins-c9tnl"
Oct 30 21:53:44.037827 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.037702    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/446beacf-70a0-4d47-96ea-bbe1ca4a5f28-host-cni-netd\") pod \"ovnkube-node-mn9vl\" (UID: \"446beacf-70a0-4d47-96ea-bbe1ca4a5f28\") " pod="openshift-ovn-kubernetes/ovnkube-node-mn9vl"
Oct 30 21:53:44.037827 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.037717    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/446beacf-70a0-4d47-96ea-bbe1ca4a5f28-env-overrides\") pod \"ovnkube-node-mn9vl\" (UID: \"446beacf-70a0-4d47-96ea-bbe1ca4a5f28\") " pod="openshift-ovn-kubernetes/ovnkube-node-mn9vl"
Oct 30 21:53:44.037827 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.037730    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5d5057fc-dd8a-4dce-b8bb-70b2dc3b074c-cnibin\") pod \"multus-jkdxt\" (UID: \"5d5057fc-dd8a-4dce-b8bb-70b2dc3b074c\") " pod="openshift-multus/multus-jkdxt"
Oct 30 21:53:44.038471 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.037746    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/af83b188-09cb-4621-a9da-be33856d19f0-dbus\") pod \"global-pull-secret-syncer-q7fwz\" (UID: \"af83b188-09cb-4621-a9da-be33856d19f0\") " pod="kube-system/global-pull-secret-syncer-q7fwz"
Oct 30 21:53:44.038471 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.037767    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/bebdb584-1a96-453d-8a6d-b332e47bec3d-etc-systemd\") pod \"tuned-h8csb\" (UID: \"bebdb584-1a96-453d-8a6d-b332e47bec3d\") " pod="openshift-cluster-node-tuning-operator/tuned-h8csb"
Oct 30 21:53:44.038471 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.037783    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/446beacf-70a0-4d47-96ea-bbe1ca4a5f28-ovn-node-metrics-cert\") pod \"ovnkube-node-mn9vl\" (UID: \"446beacf-70a0-4d47-96ea-bbe1ca4a5f28\") " pod="openshift-ovn-kubernetes/ovnkube-node-mn9vl"
Oct 30 21:53:44.038471 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.037802    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/bebdb584-1a96-453d-8a6d-b332e47bec3d-etc-sysctl-conf\") pod \"tuned-h8csb\" (UID: \"bebdb584-1a96-453d-8a6d-b332e47bec3d\") " pod="openshift-cluster-node-tuning-operator/tuned-h8csb"
Oct 30 21:53:44.038471 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.037821    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lgk4w\" (UniqueName: \"kubernetes.io/projected/15e1de31-92a1-4179-8ec1-400346880d1d-kube-api-access-lgk4w\") pod \"aws-ebs-csi-driver-node-kqbp9\" (UID: \"15e1de31-92a1-4179-8ec1-400346880d1d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kqbp9"
Oct 30 21:53:44.038471 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.037837    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/3e60fceb-8682-45cc-a472-f3d7017c3395-tuning-conf-dir\") pod \"multus-additional-cni-plugins-c9tnl\" (UID: \"3e60fceb-8682-45cc-a472-f3d7017c3395\") " pod="openshift-multus/multus-additional-cni-plugins-c9tnl"
Oct 30 21:53:44.038471 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.037851    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/2bcee03a-9b40-4155-8d81-43dece439162-agent-certs\") pod \"konnectivity-agent-thksb\" (UID: \"2bcee03a-9b40-4155-8d81-43dece439162\") " pod="kube-system/konnectivity-agent-thksb"
Oct 30 21:53:44.038471 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.037872    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ljsqq\" (UniqueName: \"kubernetes.io/projected/32ae0d42-4b76-47b7-b9d6-26b09f00ba5d-kube-api-access-ljsqq\") pod \"network-check-target-fvbz8\" (UID: \"32ae0d42-4b76-47b7-b9d6-26b09f00ba5d\") " pod="openshift-network-diagnostics/network-check-target-fvbz8"
Oct 30 21:53:44.038471 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.037897    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/446beacf-70a0-4d47-96ea-bbe1ca4a5f28-host-run-netns\") pod \"ovnkube-node-mn9vl\" (UID: \"446beacf-70a0-4d47-96ea-bbe1ca4a5f28\") " pod="openshift-ovn-kubernetes/ovnkube-node-mn9vl"
Oct 30 21:53:44.038471 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.037921    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/446beacf-70a0-4d47-96ea-bbe1ca4a5f28-host-run-ovn-kubernetes\") pod \"ovnkube-node-mn9vl\" (UID: \"446beacf-70a0-4d47-96ea-bbe1ca4a5f28\") " pod="openshift-ovn-kubernetes/ovnkube-node-mn9vl"
Oct 30 21:53:44.038471 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.037937    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5d5057fc-dd8a-4dce-b8bb-70b2dc3b074c-system-cni-dir\") pod \"multus-jkdxt\" (UID: \"5d5057fc-dd8a-4dce-b8bb-70b2dc3b074c\") " pod="openshift-multus/multus-jkdxt"
Oct 30 21:53:44.038471 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.037954    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5d5057fc-dd8a-4dce-b8bb-70b2dc3b074c-host-var-lib-kubelet\") pod \"multus-jkdxt\" (UID: \"5d5057fc-dd8a-4dce-b8bb-70b2dc3b074c\") " pod="openshift-multus/multus-jkdxt"
Oct 30 21:53:44.038471 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.037976    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/446beacf-70a0-4d47-96ea-bbe1ca4a5f28-host-cni-bin\") pod \"ovnkube-node-mn9vl\" (UID: \"446beacf-70a0-4d47-96ea-bbe1ca4a5f28\") " pod="openshift-ovn-kubernetes/ovnkube-node-mn9vl"
Oct 30 21:53:44.038471 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.038013    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mkrsw\" (UniqueName: \"kubernetes.io/projected/446beacf-70a0-4d47-96ea-bbe1ca4a5f28-kube-api-access-mkrsw\") pod \"ovnkube-node-mn9vl\" (UID: \"446beacf-70a0-4d47-96ea-bbe1ca4a5f28\") " pod="openshift-ovn-kubernetes/ovnkube-node-mn9vl"
Oct 30 21:53:44.038471 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.038040    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nbqv6\" (UniqueName: \"kubernetes.io/projected/fe472529-1294-46a1-a666-dc00d5d37eca-kube-api-access-nbqv6\") pod \"node-ca-qzgkj\" (UID: \"fe472529-1294-46a1-a666-dc00d5d37eca\") " pod="openshift-image-registry/node-ca-qzgkj"
Oct 30 21:53:44.038471 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.038088    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/15e1de31-92a1-4179-8ec1-400346880d1d-kubelet-dir\") pod \"aws-ebs-csi-driver-node-kqbp9\" (UID: \"15e1de31-92a1-4179-8ec1-400346880d1d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kqbp9"
Oct 30 21:53:44.039308 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.038120    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/15e1de31-92a1-4179-8ec1-400346880d1d-etc-selinux\") pod \"aws-ebs-csi-driver-node-kqbp9\" (UID: \"15e1de31-92a1-4179-8ec1-400346880d1d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kqbp9"
Oct 30 21:53:44.039308 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.038149    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/446beacf-70a0-4d47-96ea-bbe1ca4a5f28-node-log\") pod \"ovnkube-node-mn9vl\" (UID: \"446beacf-70a0-4d47-96ea-bbe1ca4a5f28\") " pod="openshift-ovn-kubernetes/ovnkube-node-mn9vl"
Oct 30 21:53:44.039308 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.038186    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/bebdb584-1a96-453d-8a6d-b332e47bec3d-var-lib-kubelet\") pod \"tuned-h8csb\" (UID: \"bebdb584-1a96-453d-8a6d-b332e47bec3d\") " pod="openshift-cluster-node-tuning-operator/tuned-h8csb"
Oct 30 21:53:44.039308 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.038282    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/bebdb584-1a96-453d-8a6d-b332e47bec3d-host\") pod \"tuned-h8csb\" (UID: \"bebdb584-1a96-453d-8a6d-b332e47bec3d\") " pod="openshift-cluster-node-tuning-operator/tuned-h8csb"
Oct 30 21:53:44.039308 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.038314    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/bebdb584-1a96-453d-8a6d-b332e47bec3d-etc-sysconfig\") pod \"tuned-h8csb\" (UID: \"bebdb584-1a96-453d-8a6d-b332e47bec3d\") " pod="openshift-cluster-node-tuning-operator/tuned-h8csb"
Oct 30 21:53:44.039308 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.038377    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/bebdb584-1a96-453d-8a6d-b332e47bec3d-etc-sysctl-d\") pod \"tuned-h8csb\" (UID: \"bebdb584-1a96-453d-8a6d-b332e47bec3d\") " pod="openshift-cluster-node-tuning-operator/tuned-h8csb"
Oct 30 21:53:44.039308 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.038404    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/fc4a67d9-8c7c-4f71-9a8a-2bba09329bc6-hosts-file\") pod \"node-resolver-wwb9f\" (UID: \"fc4a67d9-8c7c-4f71-9a8a-2bba09329bc6\") " pod="openshift-dns/node-resolver-wwb9f"
Oct 30 21:53:44.039308 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.038446    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bebdb584-1a96-453d-8a6d-b332e47bec3d-etc-kubernetes\") pod \"tuned-h8csb\" (UID: \"bebdb584-1a96-453d-8a6d-b332e47bec3d\") " pod="openshift-cluster-node-tuning-operator/tuned-h8csb"
Oct 30 21:53:44.039308 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.038465    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/446beacf-70a0-4d47-96ea-bbe1ca4a5f28-host-slash\") pod \"ovnkube-node-mn9vl\" (UID: \"446beacf-70a0-4d47-96ea-bbe1ca4a5f28\") " pod="openshift-ovn-kubernetes/ovnkube-node-mn9vl"
Oct 30 21:53:44.039308 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.038487    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/fe472529-1294-46a1-a666-dc00d5d37eca-host\") pod \"node-ca-qzgkj\" (UID: \"fe472529-1294-46a1-a666-dc00d5d37eca\") " pod="openshift-image-registry/node-ca-qzgkj"
Oct 30 21:53:44.039308 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.038511    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/5d5057fc-dd8a-4dce-b8bb-70b2dc3b074c-host-run-k8s-cni-cncf-io\") pod \"multus-jkdxt\" (UID: \"5d5057fc-dd8a-4dce-b8bb-70b2dc3b074c\") " pod="openshift-multus/multus-jkdxt"
Oct 30 21:53:44.039308 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.038541    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5d5057fc-dd8a-4dce-b8bb-70b2dc3b074c-host-run-netns\") pod \"multus-jkdxt\" (UID: \"5d5057fc-dd8a-4dce-b8bb-70b2dc3b074c\") " pod="openshift-multus/multus-jkdxt"
Oct 30 21:53:44.039308 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.038564    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fc4a67d9-8c7c-4f71-9a8a-2bba09329bc6-tmp-dir\") pod \"node-resolver-wwb9f\" (UID: \"fc4a67d9-8c7c-4f71-9a8a-2bba09329bc6\") " pod="openshift-dns/node-resolver-wwb9f"
Oct 30 21:53:44.039308 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.038581    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/bebdb584-1a96-453d-8a6d-b332e47bec3d-etc-modprobe-d\") pod \"tuned-h8csb\" (UID: \"bebdb584-1a96-453d-8a6d-b332e47bec3d\") " pod="openshift-cluster-node-tuning-operator/tuned-h8csb"
Oct 30 21:53:44.039308 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.038618    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/bebdb584-1a96-453d-8a6d-b332e47bec3d-lib-modules\") pod \"tuned-h8csb\" (UID: \"bebdb584-1a96-453d-8a6d-b332e47bec3d\") " pod="openshift-cluster-node-tuning-operator/tuned-h8csb"
Oct 30 21:53:44.039308 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.038656    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3e60fceb-8682-45cc-a472-f3d7017c3395-system-cni-dir\") pod \"multus-additional-cni-plugins-c9tnl\" (UID: \"3e60fceb-8682-45cc-a472-f3d7017c3395\") " pod="openshift-multus/multus-additional-cni-plugins-c9tnl"
Oct 30 21:53:44.039308 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.038690    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/613f514c-6011-4936-8df5-a59b960cd3a6-iptables-alerter-script\") pod \"iptables-alerter-kghfh\" (UID: \"613f514c-6011-4936-8df5-a59b960cd3a6\") " pod="openshift-network-operator/iptables-alerter-kghfh"
Oct 30 21:53:44.040050 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.038718    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/15e1de31-92a1-4179-8ec1-400346880d1d-registration-dir\") pod \"aws-ebs-csi-driver-node-kqbp9\" (UID: \"15e1de31-92a1-4179-8ec1-400346880d1d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kqbp9"
Oct 30 21:53:44.040050 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.038764    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/15e1de31-92a1-4179-8ec1-400346880d1d-sys-fs\") pod \"aws-ebs-csi-driver-node-kqbp9\" (UID: \"15e1de31-92a1-4179-8ec1-400346880d1d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kqbp9"
Oct 30 21:53:44.040050 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.038810    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/446beacf-70a0-4d47-96ea-bbe1ca4a5f28-run-ovn\") pod \"ovnkube-node-mn9vl\" (UID: \"446beacf-70a0-4d47-96ea-bbe1ca4a5f28\") " pod="openshift-ovn-kubernetes/ovnkube-node-mn9vl"
Oct 30 21:53:44.040050 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.038843    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bebdb584-1a96-453d-8a6d-b332e47bec3d-tmp\") pod \"tuned-h8csb\" (UID: \"bebdb584-1a96-453d-8a6d-b332e47bec3d\") " pod="openshift-cluster-node-tuning-operator/tuned-h8csb"
Oct 30 21:53:44.040050 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.038873    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/3e60fceb-8682-45cc-a472-f3d7017c3395-cnibin\") pod \"multus-additional-cni-plugins-c9tnl\" (UID: \"3e60fceb-8682-45cc-a472-f3d7017c3395\") " pod="openshift-multus/multus-additional-cni-plugins-c9tnl"
Oct 30 21:53:44.040050 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.038932    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/3e60fceb-8682-45cc-a472-f3d7017c3395-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-c9tnl\" (UID: \"3e60fceb-8682-45cc-a472-f3d7017c3395\") " pod="openshift-multus/multus-additional-cni-plugins-c9tnl"
Oct 30 21:53:44.040050 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.038969    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r6wtm\" (UniqueName: \"kubernetes.io/projected/532658b6-c86e-4feb-89cb-14a91316a7a0-kube-api-access-r6wtm\") pod \"network-metrics-daemon-n8xrh\" (UID: \"532658b6-c86e-4feb-89cb-14a91316a7a0\") " pod="openshift-multus/network-metrics-daemon-n8xrh"
Oct 30 21:53:44.040050 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.039031    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/446beacf-70a0-4d47-96ea-bbe1ca4a5f28-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-mn9vl\" (UID: \"446beacf-70a0-4d47-96ea-bbe1ca4a5f28\") " pod="openshift-ovn-kubernetes/ovnkube-node-mn9vl"
Oct 30 21:53:44.040050 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.039086    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5d5057fc-dd8a-4dce-b8bb-70b2dc3b074c-multus-cni-dir\") pod \"multus-jkdxt\" (UID: \"5d5057fc-dd8a-4dce-b8bb-70b2dc3b074c\") " pod="openshift-multus/multus-jkdxt"
Oct 30 21:53:44.040050 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.039129    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/5d5057fc-dd8a-4dce-b8bb-70b2dc3b074c-multus-daemon-config\") pod \"multus-jkdxt\" (UID: \"5d5057fc-dd8a-4dce-b8bb-70b2dc3b074c\") " pod="openshift-multus/multus-jkdxt"
Oct 30 21:53:44.040050 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.039173    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h6b6r\" (UniqueName: \"kubernetes.io/projected/613f514c-6011-4936-8df5-a59b960cd3a6-kube-api-access-h6b6r\") pod \"iptables-alerter-kghfh\" (UID: \"613f514c-6011-4936-8df5-a59b960cd3a6\") " pod="openshift-network-operator/iptables-alerter-kghfh"
Oct 30 21:53:44.040050 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.039201    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/15e1de31-92a1-4179-8ec1-400346880d1d-socket-dir\") pod \"aws-ebs-csi-driver-node-kqbp9\" (UID: \"15e1de31-92a1-4179-8ec1-400346880d1d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kqbp9"
Oct 30 21:53:44.040050 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.039230    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/5d5057fc-dd8a-4dce-b8bb-70b2dc3b074c-multus-socket-dir-parent\") pod \"multus-jkdxt\" (UID: \"5d5057fc-dd8a-4dce-b8bb-70b2dc3b074c\") " pod="openshift-multus/multus-jkdxt"
Oct 30 21:53:44.083567 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.083537    2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Oct 30 21:53:44.083770 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.083750    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Oct 30 21:53:44.083994 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.083978    2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-hxntv\""
Oct 30 21:53:44.084098 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.084080    2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-node-identity-dockercfg-h8qq9\""
Oct 30 21:53:44.084153 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.084128    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Oct 30 21:53:44.084460 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.084437    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Oct 30 21:53:44.084460 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.084444    2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-8hlnq\""
Oct 30 21:53:44.084637 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.084461    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Oct 30 21:53:44.084637 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.084512    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Oct 30 21:53:44.084637 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.084608    2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-vv6qp\""
Oct 30 21:53:44.084637 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.084613    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Oct 30 21:53:44.084805 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.084648    2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent-dockercfg-gzm2w\""
Oct 30 21:53:44.084805 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.084735    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Oct 30 21:53:44.084805 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.084762    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Oct 30 21:53:44.084805 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.084788    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Oct 30 21:53:44.084805 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.084790    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Oct 30 21:53:44.085007 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.084804    2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-cn5n2\""
Oct 30 21:53:44.085007 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.084821    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Oct 30 21:53:44.085007 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.084827    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Oct 30 21:53:44.085007 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.084811    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Oct 30 21:53:44.085007 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.084844    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Oct 30 21:53:44.085007 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.084856    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Oct 30 21:53:44.085007 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.084893    2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Oct 30 21:53:44.085007 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.084904    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Oct 30 21:53:44.139550 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.139523    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/3e60fceb-8682-45cc-a472-f3d7017c3395-cnibin\") pod \"multus-additional-cni-plugins-c9tnl\" (UID: \"3e60fceb-8682-45cc-a472-f3d7017c3395\") " pod="openshift-multus/multus-additional-cni-plugins-c9tnl"
Oct 30 21:53:44.139691 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.139562    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/3e60fceb-8682-45cc-a472-f3d7017c3395-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-c9tnl\" (UID: \"3e60fceb-8682-45cc-a472-f3d7017c3395\") " pod="openshift-multus/multus-additional-cni-plugins-c9tnl"
Oct 30 21:53:44.139691 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.139592    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r6wtm\" (UniqueName: \"kubernetes.io/projected/532658b6-c86e-4feb-89cb-14a91316a7a0-kube-api-access-r6wtm\") pod \"network-metrics-daemon-n8xrh\" (UID: \"532658b6-c86e-4feb-89cb-14a91316a7a0\") " pod="openshift-multus/network-metrics-daemon-n8xrh"
Oct 30 21:53:44.139691 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.139660    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/3e60fceb-8682-45cc-a472-f3d7017c3395-cnibin\") pod \"multus-additional-cni-plugins-c9tnl\" (UID: \"3e60fceb-8682-45cc-a472-f3d7017c3395\") " pod="openshift-multus/multus-additional-cni-plugins-c9tnl"
Oct 30 21:53:44.139857 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.139717    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/446beacf-70a0-4d47-96ea-bbe1ca4a5f28-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-mn9vl\" (UID: \"446beacf-70a0-4d47-96ea-bbe1ca4a5f28\") " pod="openshift-ovn-kubernetes/ovnkube-node-mn9vl"
Oct 30 21:53:44.139857 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.139773    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5d5057fc-dd8a-4dce-b8bb-70b2dc3b074c-multus-cni-dir\") pod \"multus-jkdxt\" (UID: \"5d5057fc-dd8a-4dce-b8bb-70b2dc3b074c\") " pod="openshift-multus/multus-jkdxt"
Oct 30 21:53:44.139857 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.139814    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/5d5057fc-dd8a-4dce-b8bb-70b2dc3b074c-multus-daemon-config\") pod \"multus-jkdxt\" (UID: \"5d5057fc-dd8a-4dce-b8bb-70b2dc3b074c\") " pod="openshift-multus/multus-jkdxt"
Oct 30 21:53:44.139857 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.139843    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h6b6r\" (UniqueName: \"kubernetes.io/projected/613f514c-6011-4936-8df5-a59b960cd3a6-kube-api-access-h6b6r\") pod \"iptables-alerter-kghfh\" (UID: \"613f514c-6011-4936-8df5-a59b960cd3a6\") " pod="openshift-network-operator/iptables-alerter-kghfh"
Oct 30 21:53:44.140176 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.139872    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/15e1de31-92a1-4179-8ec1-400346880d1d-socket-dir\") pod \"aws-ebs-csi-driver-node-kqbp9\" (UID: \"15e1de31-92a1-4179-8ec1-400346880d1d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kqbp9"
Oct 30 21:53:44.140176 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.139896    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/5d5057fc-dd8a-4dce-b8bb-70b2dc3b074c-multus-socket-dir-parent\") pod \"multus-jkdxt\" (UID: \"5d5057fc-dd8a-4dce-b8bb-70b2dc3b074c\") " pod="openshift-multus/multus-jkdxt"
Oct 30 21:53:44.140176 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.139921    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/af83b188-09cb-4621-a9da-be33856d19f0-kubelet-config\") pod \"global-pull-secret-syncer-q7fwz\" (UID: \"af83b188-09cb-4621-a9da-be33856d19f0\") " pod="kube-system/global-pull-secret-syncer-q7fwz"
Oct 30 21:53:44.140176 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.139946    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/af83b188-09cb-4621-a9da-be33856d19f0-original-pull-secret\") pod \"global-pull-secret-syncer-q7fwz\" (UID: \"af83b188-09cb-4621-a9da-be33856d19f0\") " pod="kube-system/global-pull-secret-syncer-q7fwz"
Oct 30 21:53:44.140176 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.139967    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r4qz4\" (UniqueName: \"kubernetes.io/projected/bebdb584-1a96-453d-8a6d-b332e47bec3d-kube-api-access-r4qz4\") pod \"tuned-h8csb\" (UID: \"bebdb584-1a96-453d-8a6d-b332e47bec3d\") " pod="openshift-cluster-node-tuning-operator/tuned-h8csb"
Oct 30 21:53:44.140176 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.139983    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/15e1de31-92a1-4179-8ec1-400346880d1d-device-dir\") pod \"aws-ebs-csi-driver-node-kqbp9\" (UID: \"15e1de31-92a1-4179-8ec1-400346880d1d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kqbp9"
Oct 30 21:53:44.140176 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.140000    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/446beacf-70a0-4d47-96ea-bbe1ca4a5f28-run-systemd\") pod \"ovnkube-node-mn9vl\" (UID: \"446beacf-70a0-4d47-96ea-bbe1ca4a5f28\") " pod="openshift-ovn-kubernetes/ovnkube-node-mn9vl"
Oct 30 21:53:44.140176 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.140033    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/446beacf-70a0-4d47-96ea-bbe1ca4a5f28-log-socket\") pod \"ovnkube-node-mn9vl\" (UID: \"446beacf-70a0-4d47-96ea-bbe1ca4a5f28\") " pod="openshift-ovn-kubernetes/ovnkube-node-mn9vl"
Oct 30 21:53:44.140176 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.140053    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5d5057fc-dd8a-4dce-b8bb-70b2dc3b074c-cni-binary-copy\") pod \"multus-jkdxt\" (UID: \"5d5057fc-dd8a-4dce-b8bb-70b2dc3b074c\") " pod="openshift-multus/multus-jkdxt"
Oct 30 21:53:44.140176 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.140061    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5d5057fc-dd8a-4dce-b8bb-70b2dc3b074c-multus-cni-dir\") pod \"multus-jkdxt\" (UID: \"5d5057fc-dd8a-4dce-b8bb-70b2dc3b074c\") " pod="openshift-multus/multus-jkdxt"
Oct 30 21:53:44.140176 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.140077    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/5d5057fc-dd8a-4dce-b8bb-70b2dc3b074c-hostroot\") pod \"multus-jkdxt\" (UID: \"5d5057fc-dd8a-4dce-b8bb-70b2dc3b074c\") " pod="openshift-multus/multus-jkdxt"
Oct 30 21:53:44.140176 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.140089    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/5d5057fc-dd8a-4dce-b8bb-70b2dc3b074c-multus-socket-dir-parent\") pod \"multus-jkdxt\" (UID: \"5d5057fc-dd8a-4dce-b8bb-70b2dc3b074c\") " pod="openshift-multus/multus-jkdxt"
Oct 30 21:53:44.140176 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.140130    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/15e1de31-92a1-4179-8ec1-400346880d1d-device-dir\") pod \"aws-ebs-csi-driver-node-kqbp9\" (UID: \"15e1de31-92a1-4179-8ec1-400346880d1d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kqbp9"
Oct 30 21:53:44.140176 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.140138    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/5d5057fc-dd8a-4dce-b8bb-70b2dc3b074c-hostroot\") pod \"multus-jkdxt\" (UID: \"5d5057fc-dd8a-4dce-b8bb-70b2dc3b074c\") " pod="openshift-multus/multus-jkdxt"
Oct 30 21:53:44.140176 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.140184    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/446beacf-70a0-4d47-96ea-bbe1ca4a5f28-run-systemd\") pod \"ovnkube-node-mn9vl\" (UID: \"446beacf-70a0-4d47-96ea-bbe1ca4a5f28\") " pod="openshift-ovn-kubernetes/ovnkube-node-mn9vl"
Oct 30 21:53:44.140840 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.140167    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/532658b6-c86e-4feb-89cb-14a91316a7a0-metrics-certs\") pod \"network-metrics-daemon-n8xrh\" (UID: \"532658b6-c86e-4feb-89cb-14a91316a7a0\") " pod="openshift-multus/network-metrics-daemon-n8xrh"
Oct 30 21:53:44.140840 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.140217    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/446beacf-70a0-4d47-96ea-bbe1ca4a5f28-log-socket\") pod \"ovnkube-node-mn9vl\" (UID: \"446beacf-70a0-4d47-96ea-bbe1ca4a5f28\") " pod="openshift-ovn-kubernetes/ovnkube-node-mn9vl"
Oct 30 21:53:44.140840 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.140232    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/446beacf-70a0-4d47-96ea-bbe1ca4a5f28-systemd-units\") pod \"ovnkube-node-mn9vl\" (UID: \"446beacf-70a0-4d47-96ea-bbe1ca4a5f28\") " pod="openshift-ovn-kubernetes/ovnkube-node-mn9vl"
Oct 30 21:53:44.140840 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.140257    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/446beacf-70a0-4d47-96ea-bbe1ca4a5f28-ovnkube-config\") pod \"ovnkube-node-mn9vl\" (UID: \"446beacf-70a0-4d47-96ea-bbe1ca4a5f28\") " pod="openshift-ovn-kubernetes/ovnkube-node-mn9vl"
Oct 30 21:53:44.140840 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.140283    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/5d5057fc-dd8a-4dce-b8bb-70b2dc3b074c-host-var-lib-cni-multus\") pod \"multus-jkdxt\" (UID: \"5d5057fc-dd8a-4dce-b8bb-70b2dc3b074c\") " pod="openshift-multus/multus-jkdxt"
Oct 30 21:53:44.140840 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.140306    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5d5057fc-dd8a-4dce-b8bb-70b2dc3b074c-multus-conf-dir\") pod \"multus-jkdxt\" (UID: \"5d5057fc-dd8a-4dce-b8bb-70b2dc3b074c\") " pod="openshift-multus/multus-jkdxt"
Oct 30 21:53:44.140840 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.140357    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5d5057fc-dd8a-4dce-b8bb-70b2dc3b074c-multus-conf-dir\") pod \"multus-jkdxt\" (UID: \"5d5057fc-dd8a-4dce-b8bb-70b2dc3b074c\") " pod="openshift-multus/multus-jkdxt"
Oct 30 21:53:44.140840 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.140418    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/af83b188-09cb-4621-a9da-be33856d19f0-kubelet-config\") pod \"global-pull-secret-syncer-q7fwz\" (UID: \"af83b188-09cb-4621-a9da-be33856d19f0\") " pod="kube-system/global-pull-secret-syncer-q7fwz"
Oct 30 21:53:44.140840 ip-10-0-143-246 kubenswrapper[2577]: E1030 21:53:44.140498    2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 30 21:53:44.140840 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.140560    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/bebdb584-1a96-453d-8a6d-b332e47bec3d-run\") pod \"tuned-h8csb\" (UID: \"bebdb584-1a96-453d-8a6d-b332e47bec3d\") " pod="openshift-cluster-node-tuning-operator/tuned-h8csb"
Oct 30 21:53:44.140840 ip-10-0-143-246 kubenswrapper[2577]: E1030 21:53:44.140577    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/af83b188-09cb-4621-a9da-be33856d19f0-original-pull-secret podName:af83b188-09cb-4621-a9da-be33856d19f0 nodeName:}" failed. No retries permitted until 2026-10-30 21:53:44.640553726 +0000 UTC m=+2.202966573 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/af83b188-09cb-4621-a9da-be33856d19f0-original-pull-secret") pod "global-pull-secret-syncer-q7fwz" (UID: "af83b188-09cb-4621-a9da-be33856d19f0") : object "kube-system"/"original-pull-secret" not registered
Oct 30 21:53:44.140840 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.140580    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/15e1de31-92a1-4179-8ec1-400346880d1d-socket-dir\") pod \"aws-ebs-csi-driver-node-kqbp9\" (UID: \"15e1de31-92a1-4179-8ec1-400346880d1d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kqbp9"
Oct 30 21:53:44.140840 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.140604    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/bebdb584-1a96-453d-8a6d-b332e47bec3d-sys\") pod \"tuned-h8csb\" (UID: \"bebdb584-1a96-453d-8a6d-b332e47bec3d\") " pod="openshift-cluster-node-tuning-operator/tuned-h8csb"
Oct 30 21:53:44.140840 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.140614    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/bebdb584-1a96-453d-8a6d-b332e47bec3d-run\") pod \"tuned-h8csb\" (UID: \"bebdb584-1a96-453d-8a6d-b332e47bec3d\") " pod="openshift-cluster-node-tuning-operator/tuned-h8csb"
Oct 30 21:53:44.140840 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.140634    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/3e60fceb-8682-45cc-a472-f3d7017c3395-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-c9tnl\" (UID: \"3e60fceb-8682-45cc-a472-f3d7017c3395\") " pod="openshift-multus/multus-additional-cni-plugins-c9tnl"
Oct 30 21:53:44.140840 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.140661    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/446beacf-70a0-4d47-96ea-bbe1ca4a5f28-host-kubelet\") pod \"ovnkube-node-mn9vl\" (UID: \"446beacf-70a0-4d47-96ea-bbe1ca4a5f28\") " pod="openshift-ovn-kubernetes/ovnkube-node-mn9vl"
Oct 30 21:53:44.140840 ip-10-0-143-246 kubenswrapper[2577]: E1030 21:53:44.140687    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 21:53:44.140840 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.140701    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/bebdb584-1a96-453d-8a6d-b332e47bec3d-sys\") pod \"tuned-h8csb\" (UID: \"bebdb584-1a96-453d-8a6d-b332e47bec3d\") " pod="openshift-cluster-node-tuning-operator/tuned-h8csb"
Oct 30 21:53:44.141582 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.140726    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/446beacf-70a0-4d47-96ea-bbe1ca4a5f28-etc-openvswitch\") pod \"ovnkube-node-mn9vl\" (UID: \"446beacf-70a0-4d47-96ea-bbe1ca4a5f28\") " pod="openshift-ovn-kubernetes/ovnkube-node-mn9vl"
Oct 30 21:53:44.141582 ip-10-0-143-246 kubenswrapper[2577]: E1030 21:53:44.140739    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/532658b6-c86e-4feb-89cb-14a91316a7a0-metrics-certs podName:532658b6-c86e-4feb-89cb-14a91316a7a0 nodeName:}" failed. No retries permitted until 2026-10-30 21:53:44.640722034 +0000 UTC m=+2.203134866 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/532658b6-c86e-4feb-89cb-14a91316a7a0-metrics-certs") pod "network-metrics-daemon-n8xrh" (UID: "532658b6-c86e-4feb-89cb-14a91316a7a0") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 21:53:44.141582 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.140663    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/446beacf-70a0-4d47-96ea-bbe1ca4a5f28-systemd-units\") pod \"ovnkube-node-mn9vl\" (UID: \"446beacf-70a0-4d47-96ea-bbe1ca4a5f28\") " pod="openshift-ovn-kubernetes/ovnkube-node-mn9vl"
Oct 30 21:53:44.141582 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.140690    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/446beacf-70a0-4d47-96ea-bbe1ca4a5f28-etc-openvswitch\") pod \"ovnkube-node-mn9vl\" (UID: \"446beacf-70a0-4d47-96ea-bbe1ca4a5f28\") " pod="openshift-ovn-kubernetes/ovnkube-node-mn9vl"
Oct 30 21:53:44.141582 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.140789    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/5d5057fc-dd8a-4dce-b8bb-70b2dc3b074c-host-var-lib-cni-multus\") pod \"multus-jkdxt\" (UID: \"5d5057fc-dd8a-4dce-b8bb-70b2dc3b074c\") " pod="openshift-multus/multus-jkdxt"
Oct 30 21:53:44.141582 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.140801    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5d5057fc-dd8a-4dce-b8bb-70b2dc3b074c-etc-kubernetes\") pod \"multus-jkdxt\" (UID: \"5d5057fc-dd8a-4dce-b8bb-70b2dc3b074c\") " pod="openshift-multus/multus-jkdxt"
Oct 30 21:53:44.141582 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.140804    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/3e60fceb-8682-45cc-a472-f3d7017c3395-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-c9tnl\" (UID: \"3e60fceb-8682-45cc-a472-f3d7017c3395\") " pod="openshift-multus/multus-additional-cni-plugins-c9tnl"
Oct 30 21:53:44.141582 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.140833    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5d5057fc-dd8a-4dce-b8bb-70b2dc3b074c-etc-kubernetes\") pod \"multus-jkdxt\" (UID: \"5d5057fc-dd8a-4dce-b8bb-70b2dc3b074c\") " pod="openshift-multus/multus-jkdxt"
Oct 30 21:53:44.141582 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.140833    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/2bcee03a-9b40-4155-8d81-43dece439162-konnectivity-ca\") pod \"konnectivity-agent-thksb\" (UID: \"2bcee03a-9b40-4155-8d81-43dece439162\") " pod="kube-system/konnectivity-agent-thksb"
Oct 30 21:53:44.141582 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.140873    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/3e60fceb-8682-45cc-a472-f3d7017c3395-cni-binary-copy\") pod \"multus-additional-cni-plugins-c9tnl\" (UID: \"3e60fceb-8682-45cc-a472-f3d7017c3395\") " pod="openshift-multus/multus-additional-cni-plugins-c9tnl"
Oct 30 21:53:44.141582 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.140904    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jp25c\" (UniqueName: \"kubernetes.io/projected/fc4a67d9-8c7c-4f71-9a8a-2bba09329bc6-kube-api-access-jp25c\") pod \"node-resolver-wwb9f\" (UID: \"fc4a67d9-8c7c-4f71-9a8a-2bba09329bc6\") " pod="openshift-dns/node-resolver-wwb9f"
Oct 30 21:53:44.141582 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.140927    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/446beacf-70a0-4d47-96ea-bbe1ca4a5f28-ovnkube-script-lib\") pod \"ovnkube-node-mn9vl\" (UID: \"446beacf-70a0-4d47-96ea-bbe1ca4a5f28\") " pod="openshift-ovn-kubernetes/ovnkube-node-mn9vl"
Oct 30 21:53:44.141582 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.140952    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/fe472529-1294-46a1-a666-dc00d5d37eca-serviceca\") pod \"node-ca-qzgkj\" (UID: \"fe472529-1294-46a1-a666-dc00d5d37eca\") " pod="openshift-image-registry/node-ca-qzgkj"
Oct 30 21:53:44.141582 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.140978    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5d5057fc-dd8a-4dce-b8bb-70b2dc3b074c-host-var-lib-cni-bin\") pod \"multus-jkdxt\" (UID: \"5d5057fc-dd8a-4dce-b8bb-70b2dc3b074c\") " pod="openshift-multus/multus-jkdxt"
Oct 30 21:53:44.141582 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.141006    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/446beacf-70a0-4d47-96ea-bbe1ca4a5f28-run-openvswitch\") pod \"ovnkube-node-mn9vl\" (UID: \"446beacf-70a0-4d47-96ea-bbe1ca4a5f28\") " pod="openshift-ovn-kubernetes/ovnkube-node-mn9vl"
Oct 30 21:53:44.141582 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.141054    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kncjh\" (UniqueName: \"kubernetes.io/projected/5d5057fc-dd8a-4dce-b8bb-70b2dc3b074c-kube-api-access-kncjh\") pod \"multus-jkdxt\" (UID: \"5d5057fc-dd8a-4dce-b8bb-70b2dc3b074c\") " pod="openshift-multus/multus-jkdxt"
Oct 30 21:53:44.141582 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.141081    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/bebdb584-1a96-453d-8a6d-b332e47bec3d-etc-tuned\") pod \"tuned-h8csb\" (UID: \"bebdb584-1a96-453d-8a6d-b332e47bec3d\") " pod="openshift-cluster-node-tuning-operator/tuned-h8csb"
Oct 30 21:53:44.142290 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.141105    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/613f514c-6011-4936-8df5-a59b960cd3a6-host-slash\") pod \"iptables-alerter-kghfh\" (UID: \"613f514c-6011-4936-8df5-a59b960cd3a6\") " pod="openshift-network-operator/iptables-alerter-kghfh"
Oct 30 21:53:44.142290 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.141130    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/446beacf-70a0-4d47-96ea-bbe1ca4a5f28-var-lib-openvswitch\") pod \"ovnkube-node-mn9vl\" (UID: \"446beacf-70a0-4d47-96ea-bbe1ca4a5f28\") " pod="openshift-ovn-kubernetes/ovnkube-node-mn9vl"
Oct 30 21:53:44.142290 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.141165    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/5d5057fc-dd8a-4dce-b8bb-70b2dc3b074c-host-run-multus-certs\") pod \"multus-jkdxt\" (UID: \"5d5057fc-dd8a-4dce-b8bb-70b2dc3b074c\") " pod="openshift-multus/multus-jkdxt"
Oct 30 21:53:44.142290 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.141192    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6bsvp\" (UniqueName: \"kubernetes.io/projected/3e60fceb-8682-45cc-a472-f3d7017c3395-kube-api-access-6bsvp\") pod \"multus-additional-cni-plugins-c9tnl\" (UID: \"3e60fceb-8682-45cc-a472-f3d7017c3395\") " pod="openshift-multus/multus-additional-cni-plugins-c9tnl"
Oct 30 21:53:44.142290 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.141218    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/446beacf-70a0-4d47-96ea-bbe1ca4a5f28-host-cni-netd\") pod \"ovnkube-node-mn9vl\" (UID: \"446beacf-70a0-4d47-96ea-bbe1ca4a5f28\") " pod="openshift-ovn-kubernetes/ovnkube-node-mn9vl"
Oct 30 21:53:44.142290 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.141244    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/446beacf-70a0-4d47-96ea-bbe1ca4a5f28-env-overrides\") pod \"ovnkube-node-mn9vl\" (UID: \"446beacf-70a0-4d47-96ea-bbe1ca4a5f28\") " pod="openshift-ovn-kubernetes/ovnkube-node-mn9vl"
Oct 30 21:53:44.142290 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.141269    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5d5057fc-dd8a-4dce-b8bb-70b2dc3b074c-cnibin\") pod \"multus-jkdxt\" (UID: \"5d5057fc-dd8a-4dce-b8bb-70b2dc3b074c\") " pod="openshift-multus/multus-jkdxt"
Oct 30 21:53:44.142290 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.141294    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/af83b188-09cb-4621-a9da-be33856d19f0-dbus\") pod \"global-pull-secret-syncer-q7fwz\" (UID: \"af83b188-09cb-4621-a9da-be33856d19f0\") " pod="kube-system/global-pull-secret-syncer-q7fwz"
Oct 30 21:53:44.142290 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.141317    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/bebdb584-1a96-453d-8a6d-b332e47bec3d-etc-systemd\") pod \"tuned-h8csb\" (UID: \"bebdb584-1a96-453d-8a6d-b332e47bec3d\") " pod="openshift-cluster-node-tuning-operator/tuned-h8csb"
Oct 30 21:53:44.142290 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.141335    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/446beacf-70a0-4d47-96ea-bbe1ca4a5f28-ovn-node-metrics-cert\") pod \"ovnkube-node-mn9vl\" (UID: \"446beacf-70a0-4d47-96ea-bbe1ca4a5f28\") " pod="openshift-ovn-kubernetes/ovnkube-node-mn9vl"
Oct 30 21:53:44.142290 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.141350    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/bebdb584-1a96-453d-8a6d-b332e47bec3d-etc-sysctl-conf\") pod \"tuned-h8csb\" (UID: \"bebdb584-1a96-453d-8a6d-b332e47bec3d\") " pod="openshift-cluster-node-tuning-operator/tuned-h8csb"
Oct 30 21:53:44.142290 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.141368    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lgk4w\" (UniqueName: \"kubernetes.io/projected/15e1de31-92a1-4179-8ec1-400346880d1d-kube-api-access-lgk4w\") pod \"aws-ebs-csi-driver-node-kqbp9\" (UID: \"15e1de31-92a1-4179-8ec1-400346880d1d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kqbp9"
Oct 30 21:53:44.142290 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.141384    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/3e60fceb-8682-45cc-a472-f3d7017c3395-tuning-conf-dir\") pod \"multus-additional-cni-plugins-c9tnl\" (UID: \"3e60fceb-8682-45cc-a472-f3d7017c3395\") " pod="openshift-multus/multus-additional-cni-plugins-c9tnl"
Oct 30 21:53:44.142290 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.141402    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/2bcee03a-9b40-4155-8d81-43dece439162-agent-certs\") pod \"konnectivity-agent-thksb\" (UID: \"2bcee03a-9b40-4155-8d81-43dece439162\") " pod="kube-system/konnectivity-agent-thksb"
Oct 30 21:53:44.142290 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.141418    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ljsqq\" (UniqueName: \"kubernetes.io/projected/32ae0d42-4b76-47b7-b9d6-26b09f00ba5d-kube-api-access-ljsqq\") pod \"network-check-target-fvbz8\" (UID: \"32ae0d42-4b76-47b7-b9d6-26b09f00ba5d\") " pod="openshift-network-diagnostics/network-check-target-fvbz8"
Oct 30 21:53:44.142290 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.141434    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/446beacf-70a0-4d47-96ea-bbe1ca4a5f28-host-run-netns\") pod \"ovnkube-node-mn9vl\" (UID: \"446beacf-70a0-4d47-96ea-bbe1ca4a5f28\") " pod="openshift-ovn-kubernetes/ovnkube-node-mn9vl"
Oct 30 21:53:44.142290 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.141452    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/446beacf-70a0-4d47-96ea-bbe1ca4a5f28-host-run-ovn-kubernetes\") pod \"ovnkube-node-mn9vl\" (UID: \"446beacf-70a0-4d47-96ea-bbe1ca4a5f28\") " pod="openshift-ovn-kubernetes/ovnkube-node-mn9vl"
Oct 30 21:53:44.143242 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.141469    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5d5057fc-dd8a-4dce-b8bb-70b2dc3b074c-system-cni-dir\") pod \"multus-jkdxt\" (UID: \"5d5057fc-dd8a-4dce-b8bb-70b2dc3b074c\") " pod="openshift-multus/multus-jkdxt"
Oct 30 21:53:44.143242 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.141489    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5d5057fc-dd8a-4dce-b8bb-70b2dc3b074c-host-var-lib-kubelet\") pod \"multus-jkdxt\" (UID: \"5d5057fc-dd8a-4dce-b8bb-70b2dc3b074c\") " pod="openshift-multus/multus-jkdxt"
Oct 30 21:53:44.143242 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.141506    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/446beacf-70a0-4d47-96ea-bbe1ca4a5f28-host-cni-bin\") pod \"ovnkube-node-mn9vl\" (UID: \"446beacf-70a0-4d47-96ea-bbe1ca4a5f28\") " pod="openshift-ovn-kubernetes/ovnkube-node-mn9vl"
Oct 30 21:53:44.143242 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.141521    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mkrsw\" (UniqueName: \"kubernetes.io/projected/446beacf-70a0-4d47-96ea-bbe1ca4a5f28-kube-api-access-mkrsw\") pod \"ovnkube-node-mn9vl\" (UID: \"446beacf-70a0-4d47-96ea-bbe1ca4a5f28\") " pod="openshift-ovn-kubernetes/ovnkube-node-mn9vl"
Oct 30 21:53:44.143242 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.139819    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/446beacf-70a0-4d47-96ea-bbe1ca4a5f28-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-mn9vl\" (UID: \"446beacf-70a0-4d47-96ea-bbe1ca4a5f28\") " pod="openshift-ovn-kubernetes/ovnkube-node-mn9vl"
Oct 30 21:53:44.143242 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.141540    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nbqv6\" (UniqueName: \"kubernetes.io/projected/fe472529-1294-46a1-a666-dc00d5d37eca-kube-api-access-nbqv6\") pod \"node-ca-qzgkj\" (UID: \"fe472529-1294-46a1-a666-dc00d5d37eca\") " pod="openshift-image-registry/node-ca-qzgkj"
Oct 30 21:53:44.143242 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.141558    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/15e1de31-92a1-4179-8ec1-400346880d1d-kubelet-dir\") pod \"aws-ebs-csi-driver-node-kqbp9\" (UID: \"15e1de31-92a1-4179-8ec1-400346880d1d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kqbp9"
Oct 30 21:53:44.143242 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.141576    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/15e1de31-92a1-4179-8ec1-400346880d1d-etc-selinux\") pod \"aws-ebs-csi-driver-node-kqbp9\" (UID: \"15e1de31-92a1-4179-8ec1-400346880d1d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kqbp9"
Oct 30 21:53:44.143242 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.141593    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/446beacf-70a0-4d47-96ea-bbe1ca4a5f28-node-log\") pod \"ovnkube-node-mn9vl\" (UID: \"446beacf-70a0-4d47-96ea-bbe1ca4a5f28\") " pod="openshift-ovn-kubernetes/ovnkube-node-mn9vl"
Oct 30 21:53:44.143242 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.141615    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/bebdb584-1a96-453d-8a6d-b332e47bec3d-var-lib-kubelet\") pod \"tuned-h8csb\" (UID: \"bebdb584-1a96-453d-8a6d-b332e47bec3d\") " pod="openshift-cluster-node-tuning-operator/tuned-h8csb"
Oct 30 21:53:44.143242 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.141631    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/bebdb584-1a96-453d-8a6d-b332e47bec3d-host\") pod \"tuned-h8csb\" (UID: \"bebdb584-1a96-453d-8a6d-b332e47bec3d\") " pod="openshift-cluster-node-tuning-operator/tuned-h8csb"
Oct 30 21:53:44.143242 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.141645    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/bebdb584-1a96-453d-8a6d-b332e47bec3d-etc-sysconfig\") pod \"tuned-h8csb\" (UID: \"bebdb584-1a96-453d-8a6d-b332e47bec3d\") " pod="openshift-cluster-node-tuning-operator/tuned-h8csb"
Oct 30 21:53:44.143242 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.141662    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/bebdb584-1a96-453d-8a6d-b332e47bec3d-etc-sysctl-d\") pod \"tuned-h8csb\" (UID: \"bebdb584-1a96-453d-8a6d-b332e47bec3d\") " pod="openshift-cluster-node-tuning-operator/tuned-h8csb"
Oct 30 21:53:44.143242 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.141677    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/fc4a67d9-8c7c-4f71-9a8a-2bba09329bc6-hosts-file\") pod \"node-resolver-wwb9f\" (UID: \"fc4a67d9-8c7c-4f71-9a8a-2bba09329bc6\") " pod="openshift-dns/node-resolver-wwb9f"
Oct 30 21:53:44.143242 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.141693    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bebdb584-1a96-453d-8a6d-b332e47bec3d-etc-kubernetes\") pod \"tuned-h8csb\" (UID: \"bebdb584-1a96-453d-8a6d-b332e47bec3d\") " pod="openshift-cluster-node-tuning-operator/tuned-h8csb"
Oct 30 21:53:44.143242 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.141709    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/446beacf-70a0-4d47-96ea-bbe1ca4a5f28-host-slash\") pod \"ovnkube-node-mn9vl\" (UID: \"446beacf-70a0-4d47-96ea-bbe1ca4a5f28\") " pod="openshift-ovn-kubernetes/ovnkube-node-mn9vl"
Oct 30 21:53:44.143242 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.141733    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/fe472529-1294-46a1-a666-dc00d5d37eca-host\") pod \"node-ca-qzgkj\" (UID: \"fe472529-1294-46a1-a666-dc00d5d37eca\") " pod="openshift-image-registry/node-ca-qzgkj"
Oct 30 21:53:44.144138 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.141754    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/5d5057fc-dd8a-4dce-b8bb-70b2dc3b074c-host-run-k8s-cni-cncf-io\") pod \"multus-jkdxt\" (UID: \"5d5057fc-dd8a-4dce-b8bb-70b2dc3b074c\") " pod="openshift-multus/multus-jkdxt"
Oct 30 21:53:44.144138 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.141776    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5d5057fc-dd8a-4dce-b8bb-70b2dc3b074c-host-run-netns\") pod \"multus-jkdxt\" (UID: \"5d5057fc-dd8a-4dce-b8bb-70b2dc3b074c\") " pod="openshift-multus/multus-jkdxt"
Oct 30 21:53:44.144138 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.141792    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fc4a67d9-8c7c-4f71-9a8a-2bba09329bc6-tmp-dir\") pod \"node-resolver-wwb9f\" (UID: \"fc4a67d9-8c7c-4f71-9a8a-2bba09329bc6\") " pod="openshift-dns/node-resolver-wwb9f"
Oct 30 21:53:44.144138 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.141809    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/bebdb584-1a96-453d-8a6d-b332e47bec3d-etc-modprobe-d\") pod \"tuned-h8csb\" (UID: \"bebdb584-1a96-453d-8a6d-b332e47bec3d\") " pod="openshift-cluster-node-tuning-operator/tuned-h8csb"
Oct 30 21:53:44.144138 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.141825    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/bebdb584-1a96-453d-8a6d-b332e47bec3d-lib-modules\") pod \"tuned-h8csb\" (UID: \"bebdb584-1a96-453d-8a6d-b332e47bec3d\") " pod="openshift-cluster-node-tuning-operator/tuned-h8csb"
Oct 30 21:53:44.144138 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.141865    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3e60fceb-8682-45cc-a472-f3d7017c3395-system-cni-dir\") pod \"multus-additional-cni-plugins-c9tnl\" (UID: \"3e60fceb-8682-45cc-a472-f3d7017c3395\") " pod="openshift-multus/multus-additional-cni-plugins-c9tnl"
Oct 30 21:53:44.144138 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.141867    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/446beacf-70a0-4d47-96ea-bbe1ca4a5f28-ovnkube-config\") pod \"ovnkube-node-mn9vl\" (UID: \"446beacf-70a0-4d47-96ea-bbe1ca4a5f28\") " pod="openshift-ovn-kubernetes/ovnkube-node-mn9vl"
Oct 30 21:53:44.144138 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.141894    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/613f514c-6011-4936-8df5-a59b960cd3a6-iptables-alerter-script\") pod \"iptables-alerter-kghfh\" (UID: \"613f514c-6011-4936-8df5-a59b960cd3a6\") " pod="openshift-network-operator/iptables-alerter-kghfh"
Oct 30 21:53:44.144138 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.141921    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/15e1de31-92a1-4179-8ec1-400346880d1d-registration-dir\") pod \"aws-ebs-csi-driver-node-kqbp9\" (UID: \"15e1de31-92a1-4179-8ec1-400346880d1d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kqbp9"
Oct 30 21:53:44.144138 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.141949    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5d5057fc-dd8a-4dce-b8bb-70b2dc3b074c-host-var-lib-kubelet\") pod \"multus-jkdxt\" (UID: \"5d5057fc-dd8a-4dce-b8bb-70b2dc3b074c\") " pod="openshift-multus/multus-jkdxt"
Oct 30 21:53:44.144138 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.141953    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/15e1de31-92a1-4179-8ec1-400346880d1d-sys-fs\") pod \"aws-ebs-csi-driver-node-kqbp9\" (UID: \"15e1de31-92a1-4179-8ec1-400346880d1d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kqbp9"
Oct 30 21:53:44.144138 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.141987    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/3e60fceb-8682-45cc-a472-f3d7017c3395-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-c9tnl\" (UID: \"3e60fceb-8682-45cc-a472-f3d7017c3395\") " pod="openshift-multus/multus-additional-cni-plugins-c9tnl"
Oct 30 21:53:44.144138 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.142001    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/446beacf-70a0-4d47-96ea-bbe1ca4a5f28-run-ovn\") pod \"ovnkube-node-mn9vl\" (UID: \"446beacf-70a0-4d47-96ea-bbe1ca4a5f28\") " pod="openshift-ovn-kubernetes/ovnkube-node-mn9vl"
Oct 30 21:53:44.144138 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.142007    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/15e1de31-92a1-4179-8ec1-400346880d1d-sys-fs\") pod \"aws-ebs-csi-driver-node-kqbp9\" (UID: \"15e1de31-92a1-4179-8ec1-400346880d1d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kqbp9"
Oct 30 21:53:44.144138 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.142047    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/446beacf-70a0-4d47-96ea-bbe1ca4a5f28-run-ovn\") pod \"ovnkube-node-mn9vl\" (UID: \"446beacf-70a0-4d47-96ea-bbe1ca4a5f28\") " pod="openshift-ovn-kubernetes/ovnkube-node-mn9vl"
Oct 30 21:53:44.144138 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.142051    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/5d5057fc-dd8a-4dce-b8bb-70b2dc3b074c-multus-daemon-config\") pod \"multus-jkdxt\" (UID: \"5d5057fc-dd8a-4dce-b8bb-70b2dc3b074c\") " pod="openshift-multus/multus-jkdxt"
Oct 30 21:53:44.144138 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.142081    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/446beacf-70a0-4d47-96ea-bbe1ca4a5f28-host-cni-bin\") pod \"ovnkube-node-mn9vl\" (UID: \"446beacf-70a0-4d47-96ea-bbe1ca4a5f28\") " pod="openshift-ovn-kubernetes/ovnkube-node-mn9vl"
Oct 30 21:53:44.144903 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.140980    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/446beacf-70a0-4d47-96ea-bbe1ca4a5f28-host-kubelet\") pod \"ovnkube-node-mn9vl\" (UID: \"446beacf-70a0-4d47-96ea-bbe1ca4a5f28\") " pod="openshift-ovn-kubernetes/ovnkube-node-mn9vl"
Oct 30 21:53:44.144903 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.142129    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5d5057fc-dd8a-4dce-b8bb-70b2dc3b074c-host-var-lib-cni-bin\") pod \"multus-jkdxt\" (UID: \"5d5057fc-dd8a-4dce-b8bb-70b2dc3b074c\") " pod="openshift-multus/multus-jkdxt"
Oct 30 21:53:44.144903 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.142249    2577 swap_util.go:78] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Oct 30 21:53:44.144903 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.142266    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/446beacf-70a0-4d47-96ea-bbe1ca4a5f28-run-openvswitch\") pod \"ovnkube-node-mn9vl\" (UID: \"446beacf-70a0-4d47-96ea-bbe1ca4a5f28\") " pod="openshift-ovn-kubernetes/ovnkube-node-mn9vl"
Oct 30 21:53:44.144903 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.142430    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fc4a67d9-8c7c-4f71-9a8a-2bba09329bc6-tmp-dir\") pod \"node-resolver-wwb9f\" (UID: \"fc4a67d9-8c7c-4f71-9a8a-2bba09329bc6\") " pod="openshift-dns/node-resolver-wwb9f"
Oct 30 21:53:44.144903 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.142515    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/5d5057fc-dd8a-4dce-b8bb-70b2dc3b074c-host-run-k8s-cni-cncf-io\") pod \"multus-jkdxt\" (UID: \"5d5057fc-dd8a-4dce-b8bb-70b2dc3b074c\") " pod="openshift-multus/multus-jkdxt"
Oct 30 21:53:44.144903 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.142546    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5d5057fc-dd8a-4dce-b8bb-70b2dc3b074c-host-run-netns\") pod \"multus-jkdxt\" (UID: \"5d5057fc-dd8a-4dce-b8bb-70b2dc3b074c\") " pod="openshift-multus/multus-jkdxt"
Oct 30 21:53:44.144903 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.142930    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/fe472529-1294-46a1-a666-dc00d5d37eca-host\") pod \"node-ca-qzgkj\" (UID: \"fe472529-1294-46a1-a666-dc00d5d37eca\") " pod="openshift-image-registry/node-ca-qzgkj"
Oct 30 21:53:44.144903 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.142983    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/15e1de31-92a1-4179-8ec1-400346880d1d-kubelet-dir\") pod \"aws-ebs-csi-driver-node-kqbp9\" (UID: \"15e1de31-92a1-4179-8ec1-400346880d1d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kqbp9"
Oct 30 21:53:44.144903 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.143033    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bebdb584-1a96-453d-8a6d-b332e47bec3d-etc-kubernetes\") pod \"tuned-h8csb\" (UID: \"bebdb584-1a96-453d-8a6d-b332e47bec3d\") " pod="openshift-cluster-node-tuning-operator/tuned-h8csb"
Oct 30 21:53:44.144903 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.143100    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/446beacf-70a0-4d47-96ea-bbe1ca4a5f28-host-slash\") pod \"ovnkube-node-mn9vl\" (UID: \"446beacf-70a0-4d47-96ea-bbe1ca4a5f28\") " pod="openshift-ovn-kubernetes/ovnkube-node-mn9vl"
Oct 30 21:53:44.144903 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.143100    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/446beacf-70a0-4d47-96ea-bbe1ca4a5f28-ovnkube-script-lib\") pod \"ovnkube-node-mn9vl\" (UID: \"446beacf-70a0-4d47-96ea-bbe1ca4a5f28\") " pod="openshift-ovn-kubernetes/ovnkube-node-mn9vl"
Oct 30 21:53:44.144903 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.143171    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/bebdb584-1a96-453d-8a6d-b332e47bec3d-var-lib-kubelet\") pod \"tuned-h8csb\" (UID: \"bebdb584-1a96-453d-8a6d-b332e47bec3d\") " pod="openshift-cluster-node-tuning-operator/tuned-h8csb"
Oct 30 21:53:44.144903 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.143183    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/446beacf-70a0-4d47-96ea-bbe1ca4a5f28-node-log\") pod \"ovnkube-node-mn9vl\" (UID: \"446beacf-70a0-4d47-96ea-bbe1ca4a5f28\") " pod="openshift-ovn-kubernetes/ovnkube-node-mn9vl"
Oct 30 21:53:44.144903 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.143233    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/bebdb584-1a96-453d-8a6d-b332e47bec3d-host\") pod \"tuned-h8csb\" (UID: \"bebdb584-1a96-453d-8a6d-b332e47bec3d\") " pod="openshift-cluster-node-tuning-operator/tuned-h8csb"
Oct 30 21:53:44.144903 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.143254    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/bebdb584-1a96-453d-8a6d-b332e47bec3d-etc-sysconfig\") pod \"tuned-h8csb\" (UID: \"bebdb584-1a96-453d-8a6d-b332e47bec3d\") " pod="openshift-cluster-node-tuning-operator/tuned-h8csb"
Oct 30 21:53:44.144903 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.143115    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/15e1de31-92a1-4179-8ec1-400346880d1d-etc-selinux\") pod \"aws-ebs-csi-driver-node-kqbp9\" (UID: \"15e1de31-92a1-4179-8ec1-400346880d1d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kqbp9"
Oct 30 21:53:44.144903 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.143345    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/bebdb584-1a96-453d-8a6d-b332e47bec3d-etc-sysctl-d\") pod \"tuned-h8csb\" (UID: \"bebdb584-1a96-453d-8a6d-b332e47bec3d\") " pod="openshift-cluster-node-tuning-operator/tuned-h8csb"
Oct 30 21:53:44.145461 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.143380    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bebdb584-1a96-453d-8a6d-b332e47bec3d-tmp\") pod \"tuned-h8csb\" (UID: \"bebdb584-1a96-453d-8a6d-b332e47bec3d\") " pod="openshift-cluster-node-tuning-operator/tuned-h8csb"
Oct 30 21:53:44.145461 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:44.143378    2577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb7eadc83c6b2f891221807815b7cecbc.slice/crio-a1a23f1cc1936fcf6f0fbc3af4009a0cd4a17790ed28b13c1dfc5accffe8f597 WatchSource:0}: Error finding container a1a23f1cc1936fcf6f0fbc3af4009a0cd4a17790ed28b13c1dfc5accffe8f597: Status 404 returned error can't find the container with id a1a23f1cc1936fcf6f0fbc3af4009a0cd4a17790ed28b13c1dfc5accffe8f597
Oct 30 21:53:44.145461 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.143381    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/fe472529-1294-46a1-a666-dc00d5d37eca-serviceca\") pod \"node-ca-qzgkj\" (UID: \"fe472529-1294-46a1-a666-dc00d5d37eca\") " pod="openshift-image-registry/node-ca-qzgkj"
Oct 30 21:53:44.145461 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.143427    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/446beacf-70a0-4d47-96ea-bbe1ca4a5f28-host-run-netns\") pod \"ovnkube-node-mn9vl\" (UID: \"446beacf-70a0-4d47-96ea-bbe1ca4a5f28\") " pod="openshift-ovn-kubernetes/ovnkube-node-mn9vl"
Oct 30 21:53:44.145461 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.143435    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/3e60fceb-8682-45cc-a472-f3d7017c3395-cni-binary-copy\") pod \"multus-additional-cni-plugins-c9tnl\" (UID: \"3e60fceb-8682-45cc-a472-f3d7017c3395\") " pod="openshift-multus/multus-additional-cni-plugins-c9tnl"
Oct 30 21:53:44.145461 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.143501    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5d5057fc-dd8a-4dce-b8bb-70b2dc3b074c-cnibin\") pod \"multus-jkdxt\" (UID: \"5d5057fc-dd8a-4dce-b8bb-70b2dc3b074c\") " pod="openshift-multus/multus-jkdxt"
Oct 30 21:53:44.145461 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.143528    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3e60fceb-8682-45cc-a472-f3d7017c3395-system-cni-dir\") pod \"multus-additional-cni-plugins-c9tnl\" (UID: \"3e60fceb-8682-45cc-a472-f3d7017c3395\") " pod="openshift-multus/multus-additional-cni-plugins-c9tnl"
Oct 30 21:53:44.145461 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.143528    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/af83b188-09cb-4621-a9da-be33856d19f0-dbus\") pod \"global-pull-secret-syncer-q7fwz\" (UID: \"af83b188-09cb-4621-a9da-be33856d19f0\") " pod="kube-system/global-pull-secret-syncer-q7fwz"
Oct 30 21:53:44.145461 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.143577    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/bebdb584-1a96-453d-8a6d-b332e47bec3d-etc-sysctl-conf\") pod \"tuned-h8csb\" (UID: \"bebdb584-1a96-453d-8a6d-b332e47bec3d\") " pod="openshift-cluster-node-tuning-operator/tuned-h8csb"
Oct 30 21:53:44.145461 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.143580    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/5d5057fc-dd8a-4dce-b8bb-70b2dc3b074c-host-run-multus-certs\") pod \"multus-jkdxt\" (UID: \"5d5057fc-dd8a-4dce-b8bb-70b2dc3b074c\") " pod="openshift-multus/multus-jkdxt"
Oct 30 21:53:44.145461 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.143596    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/bebdb584-1a96-453d-8a6d-b332e47bec3d-etc-modprobe-d\") pod \"tuned-h8csb\" (UID: \"bebdb584-1a96-453d-8a6d-b332e47bec3d\") " pod="openshift-cluster-node-tuning-operator/tuned-h8csb"
Oct 30 21:53:44.145461 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.143685    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/bebdb584-1a96-453d-8a6d-b332e47bec3d-lib-modules\") pod \"tuned-h8csb\" (UID: \"bebdb584-1a96-453d-8a6d-b332e47bec3d\") " pod="openshift-cluster-node-tuning-operator/tuned-h8csb"
Oct 30 21:53:44.145461 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.142006    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/2bcee03a-9b40-4155-8d81-43dece439162-konnectivity-ca\") pod \"konnectivity-agent-thksb\" (UID: \"2bcee03a-9b40-4155-8d81-43dece439162\") " pod="kube-system/konnectivity-agent-thksb"
Oct 30 21:53:44.145461 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.143723    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/3e60fceb-8682-45cc-a472-f3d7017c3395-tuning-conf-dir\") pod \"multus-additional-cni-plugins-c9tnl\" (UID: \"3e60fceb-8682-45cc-a472-f3d7017c3395\") " pod="openshift-multus/multus-additional-cni-plugins-c9tnl"
Oct 30 21:53:44.145461 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.143737    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/15e1de31-92a1-4179-8ec1-400346880d1d-registration-dir\") pod \"aws-ebs-csi-driver-node-kqbp9\" (UID: \"15e1de31-92a1-4179-8ec1-400346880d1d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kqbp9"
Oct 30 21:53:44.145461 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.143766    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/446beacf-70a0-4d47-96ea-bbe1ca4a5f28-host-run-ovn-kubernetes\") pod \"ovnkube-node-mn9vl\" (UID: \"446beacf-70a0-4d47-96ea-bbe1ca4a5f28\") " pod="openshift-ovn-kubernetes/ovnkube-node-mn9vl"
Oct 30 21:53:44.145461 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.143845    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/446beacf-70a0-4d47-96ea-bbe1ca4a5f28-var-lib-openvswitch\") pod \"ovnkube-node-mn9vl\" (UID: \"446beacf-70a0-4d47-96ea-bbe1ca4a5f28\") " pod="openshift-ovn-kubernetes/ovnkube-node-mn9vl"
Oct 30 21:53:44.145948 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.143883    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/fc4a67d9-8c7c-4f71-9a8a-2bba09329bc6-hosts-file\") pod \"node-resolver-wwb9f\" (UID: \"fc4a67d9-8c7c-4f71-9a8a-2bba09329bc6\") " pod="openshift-dns/node-resolver-wwb9f"
Oct 30 21:53:44.145948 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.143901    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5d5057fc-dd8a-4dce-b8bb-70b2dc3b074c-system-cni-dir\") pod \"multus-jkdxt\" (UID: \"5d5057fc-dd8a-4dce-b8bb-70b2dc3b074c\") " pod="openshift-multus/multus-jkdxt"
Oct 30 21:53:44.145948 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.143938    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/613f514c-6011-4936-8df5-a59b960cd3a6-host-slash\") pod \"iptables-alerter-kghfh\" (UID: \"613f514c-6011-4936-8df5-a59b960cd3a6\") " pod="openshift-network-operator/iptables-alerter-kghfh"
Oct 30 21:53:44.145948 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.143972    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/446beacf-70a0-4d47-96ea-bbe1ca4a5f28-env-overrides\") pod \"ovnkube-node-mn9vl\" (UID: \"446beacf-70a0-4d47-96ea-bbe1ca4a5f28\") " pod="openshift-ovn-kubernetes/ovnkube-node-mn9vl"
Oct 30 21:53:44.145948 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.143983    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/446beacf-70a0-4d47-96ea-bbe1ca4a5f28-host-cni-netd\") pod \"ovnkube-node-mn9vl\" (UID: \"446beacf-70a0-4d47-96ea-bbe1ca4a5f28\") " pod="openshift-ovn-kubernetes/ovnkube-node-mn9vl"
Oct 30 21:53:44.145948 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.144355    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/613f514c-6011-4936-8df5-a59b960cd3a6-iptables-alerter-script\") pod \"iptables-alerter-kghfh\" (UID: \"613f514c-6011-4936-8df5-a59b960cd3a6\") " pod="openshift-network-operator/iptables-alerter-kghfh"
Oct 30 21:53:44.145948 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.144434    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5d5057fc-dd8a-4dce-b8bb-70b2dc3b074c-cni-binary-copy\") pod \"multus-jkdxt\" (UID: \"5d5057fc-dd8a-4dce-b8bb-70b2dc3b074c\") " pod="openshift-multus/multus-jkdxt"
Oct 30 21:53:44.145948 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.142059    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/bebdb584-1a96-453d-8a6d-b332e47bec3d-etc-systemd\") pod \"tuned-h8csb\" (UID: \"bebdb584-1a96-453d-8a6d-b332e47bec3d\") " pod="openshift-cluster-node-tuning-operator/tuned-h8csb"
Oct 30 21:53:44.146493 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.146472    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/bebdb584-1a96-453d-8a6d-b332e47bec3d-etc-tuned\") pod \"tuned-h8csb\" (UID: \"bebdb584-1a96-453d-8a6d-b332e47bec3d\") " pod="openshift-cluster-node-tuning-operator/tuned-h8csb"
Oct 30 21:53:44.146723 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.146704    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/446beacf-70a0-4d47-96ea-bbe1ca4a5f28-ovn-node-metrics-cert\") pod \"ovnkube-node-mn9vl\" (UID: \"446beacf-70a0-4d47-96ea-bbe1ca4a5f28\") " pod="openshift-ovn-kubernetes/ovnkube-node-mn9vl"
Oct 30 21:53:44.146785 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.146769    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/2bcee03a-9b40-4155-8d81-43dece439162-agent-certs\") pod \"konnectivity-agent-thksb\" (UID: \"2bcee03a-9b40-4155-8d81-43dece439162\") " pod="kube-system/konnectivity-agent-thksb"
Oct 30 21:53:44.148231 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.148209    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bebdb584-1a96-453d-8a6d-b332e47bec3d-tmp\") pod \"tuned-h8csb\" (UID: \"bebdb584-1a96-453d-8a6d-b332e47bec3d\") " pod="openshift-cluster-node-tuning-operator/tuned-h8csb"
Oct 30 21:53:44.150118 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:44.150090    2577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb29468cdb315509fa1bb87ee196f0663.slice/crio-a3f130547617214efc2a1c06555379109c660bad2501f887be595e3d279ea8a4 WatchSource:0}: Error finding container a3f130547617214efc2a1c06555379109c660bad2501f887be595e3d279ea8a4: Status 404 returned error can't find the container with id a3f130547617214efc2a1c06555379109c660bad2501f887be595e3d279ea8a4
Oct 30 21:53:44.156303 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.156283    2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 21:53:44.166141 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.166110    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r6wtm\" (UniqueName: \"kubernetes.io/projected/532658b6-c86e-4feb-89cb-14a91316a7a0-kube-api-access-r6wtm\") pod \"network-metrics-daemon-n8xrh\" (UID: \"532658b6-c86e-4feb-89cb-14a91316a7a0\") " pod="openshift-multus/network-metrics-daemon-n8xrh"
Oct 30 21:53:44.166257 ip-10-0-143-246 kubenswrapper[2577]: E1030 21:53:44.166234    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 30 21:53:44.166330 ip-10-0-143-246 kubenswrapper[2577]: E1030 21:53:44.166263    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 30 21:53:44.166330 ip-10-0-143-246 kubenswrapper[2577]: E1030 21:53:44.166281    2577 projected.go:196] Error preparing data for projected volume kube-api-access-ljsqq for pod openshift-network-diagnostics/network-check-target-fvbz8: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 21:53:44.166447 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.166329    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h6b6r\" (UniqueName: \"kubernetes.io/projected/613f514c-6011-4936-8df5-a59b960cd3a6-kube-api-access-h6b6r\") pod \"iptables-alerter-kghfh\" (UID: \"613f514c-6011-4936-8df5-a59b960cd3a6\") " pod="openshift-network-operator/iptables-alerter-kghfh"
Oct 30 21:53:44.166447 ip-10-0-143-246 kubenswrapper[2577]: E1030 21:53:44.166345    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/32ae0d42-4b76-47b7-b9d6-26b09f00ba5d-kube-api-access-ljsqq podName:32ae0d42-4b76-47b7-b9d6-26b09f00ba5d nodeName:}" failed. No retries permitted until 2026-10-30 21:53:44.666324144 +0000 UTC m=+2.228736994 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-ljsqq" (UniqueName: "kubernetes.io/projected/32ae0d42-4b76-47b7-b9d6-26b09f00ba5d-kube-api-access-ljsqq") pod "network-check-target-fvbz8" (UID: "32ae0d42-4b76-47b7-b9d6-26b09f00ba5d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 21:53:44.167974 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.167943    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6bsvp\" (UniqueName: \"kubernetes.io/projected/3e60fceb-8682-45cc-a472-f3d7017c3395-kube-api-access-6bsvp\") pod \"multus-additional-cni-plugins-c9tnl\" (UID: \"3e60fceb-8682-45cc-a472-f3d7017c3395\") " pod="openshift-multus/multus-additional-cni-plugins-c9tnl"
Oct 30 21:53:44.168080 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.168043    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kncjh\" (UniqueName: \"kubernetes.io/projected/5d5057fc-dd8a-4dce-b8bb-70b2dc3b074c-kube-api-access-kncjh\") pod \"multus-jkdxt\" (UID: \"5d5057fc-dd8a-4dce-b8bb-70b2dc3b074c\") " pod="openshift-multus/multus-jkdxt"
Oct 30 21:53:44.168195 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.168167    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r4qz4\" (UniqueName: \"kubernetes.io/projected/bebdb584-1a96-453d-8a6d-b332e47bec3d-kube-api-access-r4qz4\") pod \"tuned-h8csb\" (UID: \"bebdb584-1a96-453d-8a6d-b332e47bec3d\") " pod="openshift-cluster-node-tuning-operator/tuned-h8csb"
Oct 30 21:53:44.168558 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.168534    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lgk4w\" (UniqueName: \"kubernetes.io/projected/15e1de31-92a1-4179-8ec1-400346880d1d-kube-api-access-lgk4w\") pod \"aws-ebs-csi-driver-node-kqbp9\" (UID: \"15e1de31-92a1-4179-8ec1-400346880d1d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kqbp9"
Oct 30 21:53:44.168720 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.168699    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nbqv6\" (UniqueName: \"kubernetes.io/projected/fe472529-1294-46a1-a666-dc00d5d37eca-kube-api-access-nbqv6\") pod \"node-ca-qzgkj\" (UID: \"fe472529-1294-46a1-a666-dc00d5d37eca\") " pod="openshift-image-registry/node-ca-qzgkj"
Oct 30 21:53:44.169276 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.169257    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mkrsw\" (UniqueName: \"kubernetes.io/projected/446beacf-70a0-4d47-96ea-bbe1ca4a5f28-kube-api-access-mkrsw\") pod \"ovnkube-node-mn9vl\" (UID: \"446beacf-70a0-4d47-96ea-bbe1ca4a5f28\") " pod="openshift-ovn-kubernetes/ovnkube-node-mn9vl"
Oct 30 21:53:44.169489 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.169473    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jp25c\" (UniqueName: \"kubernetes.io/projected/fc4a67d9-8c7c-4f71-9a8a-2bba09329bc6-kube-api-access-jp25c\") pod \"node-resolver-wwb9f\" (UID: \"fc4a67d9-8c7c-4f71-9a8a-2bba09329bc6\") " pod="openshift-dns/node-resolver-wwb9f"
Oct 30 21:53:44.282747 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.282709    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-kghfh"
Oct 30 21:53:44.293634 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:44.293605    2577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod613f514c_6011_4936_8df5_a59b960cd3a6.slice/crio-032e17cbfd6990eb3c9bc358001153b8ae4e45b54c916d214281a29bb48fe869 WatchSource:0}: Error finding container 032e17cbfd6990eb3c9bc358001153b8ae4e45b54c916d214281a29bb48fe869: Status 404 returned error can't find the container with id 032e17cbfd6990eb3c9bc358001153b8ae4e45b54c916d214281a29bb48fe869
Oct 30 21:53:44.317709 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.317680    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kqbp9"
Oct 30 21:53:44.327083 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:44.327050    2577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod15e1de31_92a1_4179_8ec1_400346880d1d.slice/crio-4efea9f6053c04c56805d6877572edc5c4aa453e6c534f35eefcdfdf5cc9947b WatchSource:0}: Error finding container 4efea9f6053c04c56805d6877572edc5c4aa453e6c534f35eefcdfdf5cc9947b: Status 404 returned error can't find the container with id 4efea9f6053c04c56805d6877572edc5c4aa453e6c534f35eefcdfdf5cc9947b
Oct 30 21:53:44.330814 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.330794    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-wwb9f"
Oct 30 21:53:44.340149 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:44.340123    2577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfc4a67d9_8c7c_4f71_9a8a_2bba09329bc6.slice/crio-9ce524660d905f015a7f4f67394797c49fed963f767dec7a56702ccf03654c8b WatchSource:0}: Error finding container 9ce524660d905f015a7f4f67394797c49fed963f767dec7a56702ccf03654c8b: Status 404 returned error can't find the container with id 9ce524660d905f015a7f4f67394797c49fed963f767dec7a56702ccf03654c8b
Oct 30 21:53:44.352647 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.352623    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-mn9vl"
Oct 30 21:53:44.356917 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.356897    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-h8csb"
Oct 30 21:53:44.359861 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.359838    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-qzgkj"
Oct 30 21:53:44.361970 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:44.361946    2577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod446beacf_70a0_4d47_96ea_bbe1ca4a5f28.slice/crio-e963d0a97dfea14806021bbebf9320c7147bdee479c686f3a196b1ca0ed1b577 WatchSource:0}: Error finding container e963d0a97dfea14806021bbebf9320c7147bdee479c686f3a196b1ca0ed1b577: Status 404 returned error can't find the container with id e963d0a97dfea14806021bbebf9320c7147bdee479c686f3a196b1ca0ed1b577
Oct 30 21:53:44.366940 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:44.366918    2577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbebdb584_1a96_453d_8a6d_b332e47bec3d.slice/crio-2b9d462596450531b21e0299059180d7a7a296006d63852bdaf4d5363cbef3bf WatchSource:0}: Error finding container 2b9d462596450531b21e0299059180d7a7a296006d63852bdaf4d5363cbef3bf: Status 404 returned error can't find the container with id 2b9d462596450531b21e0299059180d7a7a296006d63852bdaf4d5363cbef3bf
Oct 30 21:53:44.370377 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:44.370351    2577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfe472529_1294_46a1_a666_dc00d5d37eca.slice/crio-ee565a8a0022cec61d84d7110490393c151780d6fd4c9eb513e311b6a19048d8 WatchSource:0}: Error finding container ee565a8a0022cec61d84d7110490393c151780d6fd4c9eb513e311b6a19048d8: Status 404 returned error can't find the container with id ee565a8a0022cec61d84d7110490393c151780d6fd4c9eb513e311b6a19048d8
Oct 30 21:53:44.400228 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.400200    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-c9tnl"
Oct 30 21:53:44.404336 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.404309    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-jkdxt"
Oct 30 21:53:44.410338 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.410314    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-thksb"
Oct 30 21:53:44.411807 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:44.411779    2577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3e60fceb_8682_45cc_a472_f3d7017c3395.slice/crio-210e318062886f16a6ccfb46ef82db381f07dd4d7d0518075c707f0853ed9ea9 WatchSource:0}: Error finding container 210e318062886f16a6ccfb46ef82db381f07dd4d7d0518075c707f0853ed9ea9: Status 404 returned error can't find the container with id 210e318062886f16a6ccfb46ef82db381f07dd4d7d0518075c707f0853ed9ea9
Oct 30 21:53:44.414043 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:44.414020    2577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5d5057fc_dd8a_4dce_b8bb_70b2dc3b074c.slice/crio-3cf409dd46f74bbfa1745012a0345fc7230a78cf391809e4d473e6cf35376e9e WatchSource:0}: Error finding container 3cf409dd46f74bbfa1745012a0345fc7230a78cf391809e4d473e6cf35376e9e: Status 404 returned error can't find the container with id 3cf409dd46f74bbfa1745012a0345fc7230a78cf391809e4d473e6cf35376e9e
Oct 30 21:53:44.420997 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:53:44.420975    2577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2bcee03a_9b40_4155_8d81_43dece439162.slice/crio-539e430a94bd9fa3fb11634f988f610e9eced549a966b67a5185fe5759aad92c WatchSource:0}: Error finding container 539e430a94bd9fa3fb11634f988f610e9eced549a966b67a5185fe5759aad92c: Status 404 returned error can't find the container with id 539e430a94bd9fa3fb11634f988f610e9eced549a966b67a5185fe5759aad92c
Oct 30 21:53:44.647971 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.647918    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/af83b188-09cb-4621-a9da-be33856d19f0-original-pull-secret\") pod \"global-pull-secret-syncer-q7fwz\" (UID: \"af83b188-09cb-4621-a9da-be33856d19f0\") " pod="kube-system/global-pull-secret-syncer-q7fwz"
Oct 30 21:53:44.648170 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.647978    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/532658b6-c86e-4feb-89cb-14a91316a7a0-metrics-certs\") pod \"network-metrics-daemon-n8xrh\" (UID: \"532658b6-c86e-4feb-89cb-14a91316a7a0\") " pod="openshift-multus/network-metrics-daemon-n8xrh"
Oct 30 21:53:44.648264 ip-10-0-143-246 kubenswrapper[2577]: E1030 21:53:44.648243    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 21:53:44.648371 ip-10-0-143-246 kubenswrapper[2577]: E1030 21:53:44.648359    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/532658b6-c86e-4feb-89cb-14a91316a7a0-metrics-certs podName:532658b6-c86e-4feb-89cb-14a91316a7a0 nodeName:}" failed. No retries permitted until 2026-10-30 21:53:45.648299924 +0000 UTC m=+3.210712769 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/532658b6-c86e-4feb-89cb-14a91316a7a0-metrics-certs") pod "network-metrics-daemon-n8xrh" (UID: "532658b6-c86e-4feb-89cb-14a91316a7a0") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 21:53:44.648887 ip-10-0-143-246 kubenswrapper[2577]: E1030 21:53:44.648868    2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 30 21:53:44.648979 ip-10-0-143-246 kubenswrapper[2577]: E1030 21:53:44.648926    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/af83b188-09cb-4621-a9da-be33856d19f0-original-pull-secret podName:af83b188-09cb-4621-a9da-be33856d19f0 nodeName:}" failed. No retries permitted until 2026-10-30 21:53:45.648911084 +0000 UTC m=+3.211323922 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/af83b188-09cb-4621-a9da-be33856d19f0-original-pull-secret") pod "global-pull-secret-syncer-q7fwz" (UID: "af83b188-09cb-4621-a9da-be33856d19f0") : object "kube-system"/"original-pull-secret" not registered
Oct 30 21:53:44.750242 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.749382    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ljsqq\" (UniqueName: \"kubernetes.io/projected/32ae0d42-4b76-47b7-b9d6-26b09f00ba5d-kube-api-access-ljsqq\") pod \"network-check-target-fvbz8\" (UID: \"32ae0d42-4b76-47b7-b9d6-26b09f00ba5d\") " pod="openshift-network-diagnostics/network-check-target-fvbz8"
Oct 30 21:53:44.750242 ip-10-0-143-246 kubenswrapper[2577]: E1030 21:53:44.749632    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 30 21:53:44.750242 ip-10-0-143-246 kubenswrapper[2577]: E1030 21:53:44.749654    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 30 21:53:44.750242 ip-10-0-143-246 kubenswrapper[2577]: E1030 21:53:44.749668    2577 projected.go:196] Error preparing data for projected volume kube-api-access-ljsqq for pod openshift-network-diagnostics/network-check-target-fvbz8: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 21:53:44.750242 ip-10-0-143-246 kubenswrapper[2577]: E1030 21:53:44.749732    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/32ae0d42-4b76-47b7-b9d6-26b09f00ba5d-kube-api-access-ljsqq podName:32ae0d42-4b76-47b7-b9d6-26b09f00ba5d nodeName:}" failed. No retries permitted until 2026-10-30 21:53:45.749713083 +0000 UTC m=+3.312125923 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-ljsqq" (UniqueName: "kubernetes.io/projected/32ae0d42-4b76-47b7-b9d6-26b09f00ba5d-kube-api-access-ljsqq") pod "network-check-target-fvbz8" (UID: "32ae0d42-4b76-47b7-b9d6-26b09f00ba5d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 21:53:44.872041 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.871965    2577 reflector.go:446] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 21:53:44.948586 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:44.948528    2577 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 21:53:45.015846 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:45.015724    2577 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-10-29 21:48:44 +0000 UTC" deadline="2028-04-03 22:19:36.818156424 +0000 UTC"
Oct 30 21:53:45.015846 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:45.015794    2577 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12504h25m51.802367283s"
Oct 30 21:53:45.078177 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:45.078016    2577 reflector.go:446] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 21:53:45.107382 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:45.107305    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-jkdxt" event={"ID":"5d5057fc-dd8a-4dce-b8bb-70b2dc3b074c","Type":"ContainerStarted","Data":"3cf409dd46f74bbfa1745012a0345fc7230a78cf391809e4d473e6cf35376e9e"}
Oct 30 21:53:45.119814 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:45.119769    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-c9tnl" event={"ID":"3e60fceb-8682-45cc-a472-f3d7017c3395","Type":"ContainerStarted","Data":"210e318062886f16a6ccfb46ef82db381f07dd4d7d0518075c707f0853ed9ea9"}
Oct 30 21:53:45.127327 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:45.127178    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-h8csb" event={"ID":"bebdb584-1a96-453d-8a6d-b332e47bec3d","Type":"ContainerStarted","Data":"2b9d462596450531b21e0299059180d7a7a296006d63852bdaf4d5363cbef3bf"}
Oct 30 21:53:45.134092 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:45.133936    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mn9vl" event={"ID":"446beacf-70a0-4d47-96ea-bbe1ca4a5f28","Type":"ContainerStarted","Data":"e963d0a97dfea14806021bbebf9320c7147bdee479c686f3a196b1ca0ed1b577"}
Oct 30 21:53:45.137798 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:45.137679    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kqbp9" event={"ID":"15e1de31-92a1-4179-8ec1-400346880d1d","Type":"ContainerStarted","Data":"4efea9f6053c04c56805d6877572edc5c4aa453e6c534f35eefcdfdf5cc9947b"}
Oct 30 21:53:45.142797 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:45.142698    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-kghfh" event={"ID":"613f514c-6011-4936-8df5-a59b960cd3a6","Type":"ContainerStarted","Data":"032e17cbfd6990eb3c9bc358001153b8ae4e45b54c916d214281a29bb48fe869"}
Oct 30 21:53:45.145286 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:45.145251    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-246.ec2.internal" event={"ID":"b7eadc83c6b2f891221807815b7cecbc","Type":"ContainerStarted","Data":"a1a23f1cc1936fcf6f0fbc3af4009a0cd4a17790ed28b13c1dfc5accffe8f597"}
Oct 30 21:53:45.161731 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:45.161690    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-thksb" event={"ID":"2bcee03a-9b40-4155-8d81-43dece439162","Type":"ContainerStarted","Data":"539e430a94bd9fa3fb11634f988f610e9eced549a966b67a5185fe5759aad92c"}
Oct 30 21:53:45.168210 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:45.168162    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-qzgkj" event={"ID":"fe472529-1294-46a1-a666-dc00d5d37eca","Type":"ContainerStarted","Data":"ee565a8a0022cec61d84d7110490393c151780d6fd4c9eb513e311b6a19048d8"}
Oct 30 21:53:45.191281 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:45.191240    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-wwb9f" event={"ID":"fc4a67d9-8c7c-4f71-9a8a-2bba09329bc6","Type":"ContainerStarted","Data":"9ce524660d905f015a7f4f67394797c49fed963f767dec7a56702ccf03654c8b"}
Oct 30 21:53:45.202277 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:45.202237    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-246.ec2.internal" event={"ID":"b29468cdb315509fa1bb87ee196f0663","Type":"ContainerStarted","Data":"a3f130547617214efc2a1c06555379109c660bad2501f887be595e3d279ea8a4"}
Oct 30 21:53:45.658657 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:45.658527    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/af83b188-09cb-4621-a9da-be33856d19f0-original-pull-secret\") pod \"global-pull-secret-syncer-q7fwz\" (UID: \"af83b188-09cb-4621-a9da-be33856d19f0\") " pod="kube-system/global-pull-secret-syncer-q7fwz"
Oct 30 21:53:45.658657 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:45.658587    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/532658b6-c86e-4feb-89cb-14a91316a7a0-metrics-certs\") pod \"network-metrics-daemon-n8xrh\" (UID: \"532658b6-c86e-4feb-89cb-14a91316a7a0\") " pod="openshift-multus/network-metrics-daemon-n8xrh"
Oct 30 21:53:45.658917 ip-10-0-143-246 kubenswrapper[2577]: E1030 21:53:45.658770    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 21:53:45.658917 ip-10-0-143-246 kubenswrapper[2577]: E1030 21:53:45.658840    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/532658b6-c86e-4feb-89cb-14a91316a7a0-metrics-certs podName:532658b6-c86e-4feb-89cb-14a91316a7a0 nodeName:}" failed. No retries permitted until 2026-10-30 21:53:47.658818638 +0000 UTC m=+5.221231485 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/532658b6-c86e-4feb-89cb-14a91316a7a0-metrics-certs") pod "network-metrics-daemon-n8xrh" (UID: "532658b6-c86e-4feb-89cb-14a91316a7a0") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 21:53:45.659136 ip-10-0-143-246 kubenswrapper[2577]: E1030 21:53:45.659116    2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 30 21:53:45.659209 ip-10-0-143-246 kubenswrapper[2577]: E1030 21:53:45.659176    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/af83b188-09cb-4621-a9da-be33856d19f0-original-pull-secret podName:af83b188-09cb-4621-a9da-be33856d19f0 nodeName:}" failed. No retries permitted until 2026-10-30 21:53:47.659160426 +0000 UTC m=+5.221573262 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/af83b188-09cb-4621-a9da-be33856d19f0-original-pull-secret") pod "global-pull-secret-syncer-q7fwz" (UID: "af83b188-09cb-4621-a9da-be33856d19f0") : object "kube-system"/"original-pull-secret" not registered
Oct 30 21:53:45.759237 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:45.759197    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ljsqq\" (UniqueName: \"kubernetes.io/projected/32ae0d42-4b76-47b7-b9d6-26b09f00ba5d-kube-api-access-ljsqq\") pod \"network-check-target-fvbz8\" (UID: \"32ae0d42-4b76-47b7-b9d6-26b09f00ba5d\") " pod="openshift-network-diagnostics/network-check-target-fvbz8"
Oct 30 21:53:45.759894 ip-10-0-143-246 kubenswrapper[2577]: E1030 21:53:45.759421    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 30 21:53:45.759894 ip-10-0-143-246 kubenswrapper[2577]: E1030 21:53:45.759440    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 30 21:53:45.759894 ip-10-0-143-246 kubenswrapper[2577]: E1030 21:53:45.759453    2577 projected.go:196] Error preparing data for projected volume kube-api-access-ljsqq for pod openshift-network-diagnostics/network-check-target-fvbz8: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 21:53:45.759894 ip-10-0-143-246 kubenswrapper[2577]: E1030 21:53:45.759508    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/32ae0d42-4b76-47b7-b9d6-26b09f00ba5d-kube-api-access-ljsqq podName:32ae0d42-4b76-47b7-b9d6-26b09f00ba5d nodeName:}" failed. No retries permitted until 2026-10-30 21:53:47.759492107 +0000 UTC m=+5.321904943 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-ljsqq" (UniqueName: "kubernetes.io/projected/32ae0d42-4b76-47b7-b9d6-26b09f00ba5d-kube-api-access-ljsqq") pod "network-check-target-fvbz8" (UID: "32ae0d42-4b76-47b7-b9d6-26b09f00ba5d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 21:53:46.016112 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:46.016014    2577 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-10-29 21:48:44 +0000 UTC" deadline="2028-07-29 09:08:09.766794562 +0000 UTC"
Oct 30 21:53:46.016112 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:46.016063    2577 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15299h14m23.750736961s"
Oct 30 21:53:46.064235 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:46.064198    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-n8xrh"
Oct 30 21:53:46.064423 ip-10-0-143-246 kubenswrapper[2577]: E1030 21:53:46.064352    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-n8xrh" podUID="532658b6-c86e-4feb-89cb-14a91316a7a0"
Oct 30 21:53:46.064488 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:46.064436    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-q7fwz"
Oct 30 21:53:46.064541 ip-10-0-143-246 kubenswrapper[2577]: E1030 21:53:46.064510    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-q7fwz" podUID="af83b188-09cb-4621-a9da-be33856d19f0"
Oct 30 21:53:46.064597 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:46.064585    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fvbz8"
Oct 30 21:53:46.064702 ip-10-0-143-246 kubenswrapper[2577]: E1030 21:53:46.064659    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fvbz8" podUID="32ae0d42-4b76-47b7-b9d6-26b09f00ba5d"
Oct 30 21:53:47.680928 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:47.680380    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/af83b188-09cb-4621-a9da-be33856d19f0-original-pull-secret\") pod \"global-pull-secret-syncer-q7fwz\" (UID: \"af83b188-09cb-4621-a9da-be33856d19f0\") " pod="kube-system/global-pull-secret-syncer-q7fwz"
Oct 30 21:53:47.680928 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:47.680414    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/532658b6-c86e-4feb-89cb-14a91316a7a0-metrics-certs\") pod \"network-metrics-daemon-n8xrh\" (UID: \"532658b6-c86e-4feb-89cb-14a91316a7a0\") " pod="openshift-multus/network-metrics-daemon-n8xrh"
Oct 30 21:53:47.680928 ip-10-0-143-246 kubenswrapper[2577]: E1030 21:53:47.680518    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 21:53:47.680928 ip-10-0-143-246 kubenswrapper[2577]: E1030 21:53:47.680562    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/532658b6-c86e-4feb-89cb-14a91316a7a0-metrics-certs podName:532658b6-c86e-4feb-89cb-14a91316a7a0 nodeName:}" failed. No retries permitted until 2026-10-30 21:53:51.680549896 +0000 UTC m=+9.242962717 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/532658b6-c86e-4feb-89cb-14a91316a7a0-metrics-certs") pod "network-metrics-daemon-n8xrh" (UID: "532658b6-c86e-4feb-89cb-14a91316a7a0") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 21:53:47.680928 ip-10-0-143-246 kubenswrapper[2577]: E1030 21:53:47.680818    2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 30 21:53:47.680928 ip-10-0-143-246 kubenswrapper[2577]: E1030 21:53:47.680876    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/af83b188-09cb-4621-a9da-be33856d19f0-original-pull-secret podName:af83b188-09cb-4621-a9da-be33856d19f0 nodeName:}" failed. No retries permitted until 2026-10-30 21:53:51.68086058 +0000 UTC m=+9.243273418 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/af83b188-09cb-4621-a9da-be33856d19f0-original-pull-secret") pod "global-pull-secret-syncer-q7fwz" (UID: "af83b188-09cb-4621-a9da-be33856d19f0") : object "kube-system"/"original-pull-secret" not registered
Oct 30 21:53:47.781300 ip-10-0-143-246 kubenswrapper[2577]: E1030 21:53:47.781269    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 30 21:53:47.781300 ip-10-0-143-246 kubenswrapper[2577]: E1030 21:53:47.781302    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 30 21:53:47.781536 ip-10-0-143-246 kubenswrapper[2577]: E1030 21:53:47.781314    2577 projected.go:196] Error preparing data for projected volume kube-api-access-ljsqq for pod openshift-network-diagnostics/network-check-target-fvbz8: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 21:53:47.781536 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:47.781308    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ljsqq\" (UniqueName: \"kubernetes.io/projected/32ae0d42-4b76-47b7-b9d6-26b09f00ba5d-kube-api-access-ljsqq\") pod \"network-check-target-fvbz8\" (UID: \"32ae0d42-4b76-47b7-b9d6-26b09f00ba5d\") " pod="openshift-network-diagnostics/network-check-target-fvbz8"
Oct 30 21:53:47.781536 ip-10-0-143-246 kubenswrapper[2577]: E1030 21:53:47.781376    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/32ae0d42-4b76-47b7-b9d6-26b09f00ba5d-kube-api-access-ljsqq podName:32ae0d42-4b76-47b7-b9d6-26b09f00ba5d nodeName:}" failed. No retries permitted until 2026-10-30 21:53:51.781359655 +0000 UTC m=+9.343772480 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-ljsqq" (UniqueName: "kubernetes.io/projected/32ae0d42-4b76-47b7-b9d6-26b09f00ba5d-kube-api-access-ljsqq") pod "network-check-target-fvbz8" (UID: "32ae0d42-4b76-47b7-b9d6-26b09f00ba5d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 21:53:48.065738 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:48.065082    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fvbz8"
Oct 30 21:53:48.065738 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:48.065103    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-n8xrh"
Oct 30 21:53:48.065738 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:48.065082    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-q7fwz"
Oct 30 21:53:48.065738 ip-10-0-143-246 kubenswrapper[2577]: E1030 21:53:48.065223    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-n8xrh" podUID="532658b6-c86e-4feb-89cb-14a91316a7a0"
Oct 30 21:53:48.065738 ip-10-0-143-246 kubenswrapper[2577]: E1030 21:53:48.065259    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-q7fwz" podUID="af83b188-09cb-4621-a9da-be33856d19f0"
Oct 30 21:53:48.065738 ip-10-0-143-246 kubenswrapper[2577]: E1030 21:53:48.065321    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fvbz8" podUID="32ae0d42-4b76-47b7-b9d6-26b09f00ba5d"
Oct 30 21:53:48.255016 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:48.249741    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-c9tnl" event={"ID":"3e60fceb-8682-45cc-a472-f3d7017c3395","Type":"ContainerStarted","Data":"03a8dd6b8d585e7683860d1e3cd1375f2bf266003d39ed2d5ffccabf4eb09c75"}
Oct 30 21:53:48.259065 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:48.259029    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-246.ec2.internal" event={"ID":"b7eadc83c6b2f891221807815b7cecbc","Type":"ContainerStarted","Data":"067e439d4a1c32de3c77bcd42a8c171531377446c38f295e1e031ce55133e5a1"}
Oct 30 21:53:49.269147 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:49.269064    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-thksb" event={"ID":"2bcee03a-9b40-4155-8d81-43dece439162","Type":"ContainerStarted","Data":"3c93f7288e3c34b12da41ab92a52a043f7639335a26508e0228eeddb9283f26b"}
Oct 30 21:53:49.271780 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:49.271749    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-246.ec2.internal" event={"ID":"b29468cdb315509fa1bb87ee196f0663","Type":"ContainerStarted","Data":"3979d411383ce88f5fdefdffe5a734be0e811ea8fac3495a5a575f36fd3e01b1"}
Oct 30 21:53:49.275050 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:49.274355    2577 generic.go:356] "Generic (PLEG): container finished" podID="3e60fceb-8682-45cc-a472-f3d7017c3395" containerID="03a8dd6b8d585e7683860d1e3cd1375f2bf266003d39ed2d5ffccabf4eb09c75" exitCode=0
Oct 30 21:53:49.275050 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:49.274436    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-c9tnl" event={"ID":"3e60fceb-8682-45cc-a472-f3d7017c3395","Type":"ContainerDied","Data":"03a8dd6b8d585e7683860d1e3cd1375f2bf266003d39ed2d5ffccabf4eb09c75"}
Oct 30 21:53:49.278103 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:49.277860    2577 generic.go:356] "Generic (PLEG): container finished" podID="b7eadc83c6b2f891221807815b7cecbc" containerID="067e439d4a1c32de3c77bcd42a8c171531377446c38f295e1e031ce55133e5a1" exitCode=0
Oct 30 21:53:49.278103 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:49.277896    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-246.ec2.internal" event={"ID":"b7eadc83c6b2f891221807815b7cecbc","Type":"ContainerDied","Data":"067e439d4a1c32de3c77bcd42a8c171531377446c38f295e1e031ce55133e5a1"}
Oct 30 21:53:49.357414 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:49.356104    2577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/konnectivity-agent-thksb" podStartSLOduration=1.797527086 podStartE2EDuration="6.356083883s" totalImagesPullingTime="4.558556797s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:53:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:53:49.313219631 +0000 UTC m=+6.875632477" watchObservedRunningTime="2026-10-30 21:53:49.356083883 +0000 UTC m=+6.918496728"
Oct 30 21:53:49.404657 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:49.404593    2577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-246.ec2.internal" podStartSLOduration=6.404574384 podStartE2EDuration="6.404574384s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 21:53:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:53:49.404243503 +0000 UTC m=+6.966656373" watchObservedRunningTime="2026-10-30 21:53:49.404574384 +0000 UTC m=+6.966987234"
Oct 30 21:53:50.064724 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:50.064684    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fvbz8"
Oct 30 21:53:50.064894 ip-10-0-143-246 kubenswrapper[2577]: E1030 21:53:50.064813    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fvbz8" podUID="32ae0d42-4b76-47b7-b9d6-26b09f00ba5d"
Oct 30 21:53:50.064894 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:50.064890    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-n8xrh"
Oct 30 21:53:50.065012 ip-10-0-143-246 kubenswrapper[2577]: E1030 21:53:50.064973    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-n8xrh" podUID="532658b6-c86e-4feb-89cb-14a91316a7a0"
Oct 30 21:53:50.065073 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:50.065041    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-q7fwz"
Oct 30 21:53:50.065120 ip-10-0-143-246 kubenswrapper[2577]: E1030 21:53:50.065109    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-q7fwz" podUID="af83b188-09cb-4621-a9da-be33856d19f0"
Oct 30 21:53:50.285155 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:50.285065    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-qzgkj" event={"ID":"fe472529-1294-46a1-a666-dc00d5d37eca","Type":"ContainerStarted","Data":"78ad9d37350ad63cbd07701e8e6c6d9660e298cd69c1b992e3228f300b9be312"}
Oct 30 21:53:50.288094 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:50.288063    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-246.ec2.internal" event={"ID":"b7eadc83c6b2f891221807815b7cecbc","Type":"ContainerStarted","Data":"d372e3237f24097aed64d1f703399768a6d1994f3c7834ff64f75de0dd9df31c"}
Oct 30 21:53:50.379616 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:50.379062    2577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-image-registry/node-ca-qzgkj" podStartSLOduration=1.926598551 podStartE2EDuration="7.379042284s" totalImagesPullingTime="5.452443733s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 21:53:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:53:50.330877993 +0000 UTC m=+7.893290832" watchObservedRunningTime="2026-10-30 21:53:50.379042284 +0000 UTC m=+7.941455156"
Oct 30 21:53:50.379616 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:50.379483    2577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-246.ec2.internal" podStartSLOduration=7.379477999 podStartE2EDuration="7.379477999s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 21:53:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:53:50.378240338 +0000 UTC m=+7.940653191" watchObservedRunningTime="2026-10-30 21:53:50.379477999 +0000 UTC m=+7.941890841"
Oct 30 21:53:51.302072 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:51.302037    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kqbp9" event={"ID":"15e1de31-92a1-4179-8ec1-400346880d1d","Type":"ContainerStarted","Data":"2c55f07871381574539d7cfbd748aa3784ca1d89946359df5cefcde7a241d1b4"}
Oct 30 21:53:51.715192 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:51.714865    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/af83b188-09cb-4621-a9da-be33856d19f0-original-pull-secret\") pod \"global-pull-secret-syncer-q7fwz\" (UID: \"af83b188-09cb-4621-a9da-be33856d19f0\") " pod="kube-system/global-pull-secret-syncer-q7fwz"
Oct 30 21:53:51.715192 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:51.714922    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/532658b6-c86e-4feb-89cb-14a91316a7a0-metrics-certs\") pod \"network-metrics-daemon-n8xrh\" (UID: \"532658b6-c86e-4feb-89cb-14a91316a7a0\") " pod="openshift-multus/network-metrics-daemon-n8xrh"
Oct 30 21:53:51.715192 ip-10-0-143-246 kubenswrapper[2577]: E1030 21:53:51.715097    2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 30 21:53:51.715192 ip-10-0-143-246 kubenswrapper[2577]: E1030 21:53:51.715162    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/af83b188-09cb-4621-a9da-be33856d19f0-original-pull-secret podName:af83b188-09cb-4621-a9da-be33856d19f0 nodeName:}" failed. No retries permitted until 2026-10-30 21:53:59.715143572 +0000 UTC m=+17.277556407 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/af83b188-09cb-4621-a9da-be33856d19f0-original-pull-secret") pod "global-pull-secret-syncer-q7fwz" (UID: "af83b188-09cb-4621-a9da-be33856d19f0") : object "kube-system"/"original-pull-secret" not registered
Oct 30 21:53:51.715703 ip-10-0-143-246 kubenswrapper[2577]: E1030 21:53:51.715572    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 21:53:51.715703 ip-10-0-143-246 kubenswrapper[2577]: E1030 21:53:51.715646    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/532658b6-c86e-4feb-89cb-14a91316a7a0-metrics-certs podName:532658b6-c86e-4feb-89cb-14a91316a7a0 nodeName:}" failed. No retries permitted until 2026-10-30 21:53:59.715630238 +0000 UTC m=+17.278043075 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/532658b6-c86e-4feb-89cb-14a91316a7a0-metrics-certs") pod "network-metrics-daemon-n8xrh" (UID: "532658b6-c86e-4feb-89cb-14a91316a7a0") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 21:53:51.816543 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:51.815858    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ljsqq\" (UniqueName: \"kubernetes.io/projected/32ae0d42-4b76-47b7-b9d6-26b09f00ba5d-kube-api-access-ljsqq\") pod \"network-check-target-fvbz8\" (UID: \"32ae0d42-4b76-47b7-b9d6-26b09f00ba5d\") " pod="openshift-network-diagnostics/network-check-target-fvbz8"
Oct 30 21:53:51.816543 ip-10-0-143-246 kubenswrapper[2577]: E1030 21:53:51.816091    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 30 21:53:51.816543 ip-10-0-143-246 kubenswrapper[2577]: E1030 21:53:51.816117    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 30 21:53:51.816543 ip-10-0-143-246 kubenswrapper[2577]: E1030 21:53:51.816131    2577 projected.go:196] Error preparing data for projected volume kube-api-access-ljsqq for pod openshift-network-diagnostics/network-check-target-fvbz8: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 21:53:51.816543 ip-10-0-143-246 kubenswrapper[2577]: E1030 21:53:51.816192    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/32ae0d42-4b76-47b7-b9d6-26b09f00ba5d-kube-api-access-ljsqq podName:32ae0d42-4b76-47b7-b9d6-26b09f00ba5d nodeName:}" failed. No retries permitted until 2026-10-30 21:53:59.816172366 +0000 UTC m=+17.378585201 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-ljsqq" (UniqueName: "kubernetes.io/projected/32ae0d42-4b76-47b7-b9d6-26b09f00ba5d-kube-api-access-ljsqq") pod "network-check-target-fvbz8" (UID: "32ae0d42-4b76-47b7-b9d6-26b09f00ba5d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 21:53:52.064403 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:52.064358    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fvbz8"
Oct 30 21:53:52.064559 ip-10-0-143-246 kubenswrapper[2577]: E1030 21:53:52.064505    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fvbz8" podUID="32ae0d42-4b76-47b7-b9d6-26b09f00ba5d"
Oct 30 21:53:52.064934 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:52.064908    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-n8xrh"
Oct 30 21:53:52.065069 ip-10-0-143-246 kubenswrapper[2577]: E1030 21:53:52.065047    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-n8xrh" podUID="532658b6-c86e-4feb-89cb-14a91316a7a0"
Oct 30 21:53:52.065129 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:52.065100    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-q7fwz"
Oct 30 21:53:52.065205 ip-10-0-143-246 kubenswrapper[2577]: E1030 21:53:52.065178    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-q7fwz" podUID="af83b188-09cb-4621-a9da-be33856d19f0"
Oct 30 21:53:52.165396 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:52.165329    2577 plugin_watcher.go:201] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Oct 30 21:53:52.308748 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:52.308665    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kqbp9" event={"ID":"15e1de31-92a1-4179-8ec1-400346880d1d","Type":"ContainerStarted","Data":"dd2cd184c6ad17a6922e7e7d033bb4a910a3c6e6d18ec82ac4d5d41c7689d0bc"}
Oct 30 21:53:52.603943 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:52.603441    2577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-thksb"
Oct 30 21:53:52.604620 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:52.604274    2577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-thksb"
Oct 30 21:53:52.999724 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:52.999621    2577 reconciler.go:167] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-10-30T21:53:52.165352874Z","UUID":"d22aeca5-4156-4ed6-8c11-14719774dead","Handler":null,"Name":"","Endpoint":""}
Oct 30 21:53:53.003093 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:53.003070    2577 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Oct 30 21:53:53.003252 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:53.003103    2577 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Oct 30 21:53:53.312764 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:53.312731    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-jkdxt" event={"ID":"5d5057fc-dd8a-4dce-b8bb-70b2dc3b074c","Type":"ContainerStarted","Data":"4fdaa312b2faaabbb93c70b70feda6cf9f2486a01f015f0d0255dd311e1f9c4e"}
Oct 30 21:53:53.315508 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:53.315482    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-wwb9f" event={"ID":"fc4a67d9-8c7c-4f71-9a8a-2bba09329bc6","Type":"ContainerStarted","Data":"09be1431eb61ebd63e7efad2498a9081b36e2436ef4e24128dd527a7686d6588"}
Oct 30 21:53:53.362106 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:53.362049    2577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-multus/multus-jkdxt" podStartSLOduration=2.281900986 podStartE2EDuration="10.362031252s" totalImagesPullingTime="8.080130266s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 21:53:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:53:53.360911458 +0000 UTC m=+10.923324302" watchObservedRunningTime="2026-10-30 21:53:53.362031252 +0000 UTC m=+10.924444101"
Oct 30 21:53:53.412597 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:53.412402    2577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-dns/node-resolver-wwb9f" podStartSLOduration=1.920589879 podStartE2EDuration="10.412381175s" totalImagesPullingTime="8.491791296s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 21:53:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:53:53.41218401 +0000 UTC m=+10.974596876" watchObservedRunningTime="2026-10-30 21:53:53.412381175 +0000 UTC m=+10.974794021"
Oct 30 21:53:54.064342 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:54.064306    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-q7fwz"
Oct 30 21:53:54.064514 ip-10-0-143-246 kubenswrapper[2577]: E1030 21:53:54.064437    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-q7fwz" podUID="af83b188-09cb-4621-a9da-be33856d19f0"
Oct 30 21:53:54.064866 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:54.064843    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fvbz8"
Oct 30 21:53:54.065015 ip-10-0-143-246 kubenswrapper[2577]: E1030 21:53:54.064939    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fvbz8" podUID="32ae0d42-4b76-47b7-b9d6-26b09f00ba5d"
Oct 30 21:53:54.065105 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:54.065025    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-n8xrh"
Oct 30 21:53:54.065169 ip-10-0-143-246 kubenswrapper[2577]: E1030 21:53:54.065112    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-n8xrh" podUID="532658b6-c86e-4feb-89cb-14a91316a7a0"
Oct 30 21:53:54.319575 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:54.319495    2577 generic.go:356] "Generic (PLEG): container finished" podID="3e60fceb-8682-45cc-a472-f3d7017c3395" containerID="4dfa29366f9ad760161c09fb04efa3a6c62b648040fc26de207d36df85a10381" exitCode=0
Oct 30 21:53:54.320040 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:54.319575    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-c9tnl" event={"ID":"3e60fceb-8682-45cc-a472-f3d7017c3395","Type":"ContainerDied","Data":"4dfa29366f9ad760161c09fb04efa3a6c62b648040fc26de207d36df85a10381"}
Oct 30 21:53:54.322861 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:54.322825    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kqbp9" event={"ID":"15e1de31-92a1-4179-8ec1-400346880d1d","Type":"ContainerStarted","Data":"2290ffad935e1d2fb9e5137b5daa1896aa171e405713efbdf20d76777fbf5f5a"}
Oct 30 21:53:54.324829 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:54.324801    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-kghfh" event={"ID":"613f514c-6011-4936-8df5-a59b960cd3a6","Type":"ContainerStarted","Data":"ccfbeeb87890243562a5348723ab84ccd4af09fb9844690af72ca62eac1bd1ee"}
Oct 30 21:53:54.424578 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:54.424515    2577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-kghfh" podStartSLOduration=3.006270893 podStartE2EDuration="11.424494155s" totalImagesPullingTime="8.418223262s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 21:53:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:53:54.424286995 +0000 UTC m=+11.986699835" watchObservedRunningTime="2026-10-30 21:53:54.424494155 +0000 UTC m=+11.986906998"
Oct 30 21:53:54.862453 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:54.862424    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-wwb9f_fc4a67d9-8c7c-4f71-9a8a-2bba09329bc6/dns-node-resolver/0.log"
Oct 30 21:53:55.329614 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:55.329531    2577 generic.go:356] "Generic (PLEG): container finished" podID="3e60fceb-8682-45cc-a472-f3d7017c3395" containerID="7d2403a8c6e18c74faedb7c8c33fe7def1f92797a71523a996fd29c647c6778d" exitCode=0
Oct 30 21:53:55.330317 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:55.329614    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-c9tnl" event={"ID":"3e60fceb-8682-45cc-a472-f3d7017c3395","Type":"ContainerDied","Data":"7d2403a8c6e18c74faedb7c8c33fe7def1f92797a71523a996fd29c647c6778d"}
Oct 30 21:53:55.331755 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:55.331702    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-h8csb" event={"ID":"bebdb584-1a96-453d-8a6d-b332e47bec3d","Type":"ContainerStarted","Data":"7cdf019da4cc77f0e043e1e009ba0edd41871f99221eb6924ea89ae57796c736"}
Oct 30 21:53:55.390779 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:55.390726    2577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kqbp9" podStartSLOduration=3.8227138529999998 podStartE2EDuration="12.390712476s" totalImagesPullingTime="8.567998623s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 21:53:43 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:53:54.478552588 +0000 UTC m=+12.040965434" watchObservedRunningTime="2026-10-30 21:53:55.390712476 +0000 UTC m=+12.953125310"
Oct 30 21:53:55.807974 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:55.807906    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-qzgkj_fe472529-1294-46a1-a666-dc00d5d37eca/node-ca/0.log"
Oct 30 21:53:56.065097 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:56.065009    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-q7fwz"
Oct 30 21:53:56.065278 ip-10-0-143-246 kubenswrapper[2577]: E1030 21:53:56.065119    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-q7fwz" podUID="af83b188-09cb-4621-a9da-be33856d19f0"
Oct 30 21:53:56.065278 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:56.065009    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-n8xrh"
Oct 30 21:53:56.065278 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:56.065027    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fvbz8"
Oct 30 21:53:56.065278 ip-10-0-143-246 kubenswrapper[2577]: E1030 21:53:56.065242    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-n8xrh" podUID="532658b6-c86e-4feb-89cb-14a91316a7a0"
Oct 30 21:53:56.065470 ip-10-0-143-246 kubenswrapper[2577]: E1030 21:53:56.065356    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fvbz8" podUID="32ae0d42-4b76-47b7-b9d6-26b09f00ba5d"
Oct 30 21:53:57.339638 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:57.339602    2577 generic.go:356] "Generic (PLEG): container finished" podID="3e60fceb-8682-45cc-a472-f3d7017c3395" containerID="73754faf5ede0d182282006d5473437e56d30bd3321891a80c51f6031129545c" exitCode=0
Oct 30 21:53:57.340307 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:57.339675    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-c9tnl" event={"ID":"3e60fceb-8682-45cc-a472-f3d7017c3395","Type":"ContainerDied","Data":"73754faf5ede0d182282006d5473437e56d30bd3321891a80c51f6031129545c"}
Oct 30 21:53:57.414876 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:57.414815    2577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-h8csb" podStartSLOduration=4.03365832 podStartE2EDuration="14.414795876s" totalImagesPullingTime="10.381137556s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 21:53:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:53:55.46366959 +0000 UTC m=+13.026082434" watchObservedRunningTime="2026-10-30 21:53:57.414795876 +0000 UTC m=+14.977208732"
Oct 30 21:53:58.064827 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:58.064562    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-n8xrh"
Oct 30 21:53:58.065017 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:58.064577    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-q7fwz"
Oct 30 21:53:58.065017 ip-10-0-143-246 kubenswrapper[2577]: E1030 21:53:58.064935    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-n8xrh" podUID="532658b6-c86e-4feb-89cb-14a91316a7a0"
Oct 30 21:53:58.065122 ip-10-0-143-246 kubenswrapper[2577]: E1030 21:53:58.065024    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-q7fwz" podUID="af83b188-09cb-4621-a9da-be33856d19f0"
Oct 30 21:53:58.065122 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:58.064577    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fvbz8"
Oct 30 21:53:58.065226 ip-10-0-143-246 kubenswrapper[2577]: E1030 21:53:58.065128    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fvbz8" podUID="32ae0d42-4b76-47b7-b9d6-26b09f00ba5d"
Oct 30 21:53:59.347949 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:59.347900    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-c9tnl" event={"ID":"3e60fceb-8682-45cc-a472-f3d7017c3395","Type":"ContainerStarted","Data":"524aa4acadbe157d10a1ca4405db94dbf41a555bf00c84ccec93c047e8e91d67"}
Oct 30 21:53:59.350363 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:59.350124    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mn9vl" event={"ID":"446beacf-70a0-4d47-96ea-bbe1ca4a5f28","Type":"ContainerStarted","Data":"6774cc3fdff4d698956c945b1ea01357a1f9200ebca64fc745698755a03e756f"}
Oct 30 21:53:59.350467 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:59.350417    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mn9vl" event={"ID":"446beacf-70a0-4d47-96ea-bbe1ca4a5f28","Type":"ContainerStarted","Data":"798cc2bb748b8cca01a4aa680723bb8575442e3334849f200d1edd7146951fac"}
Oct 30 21:53:59.350467 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:59.350459    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mn9vl" event={"ID":"446beacf-70a0-4d47-96ea-bbe1ca4a5f28","Type":"ContainerStarted","Data":"c86fdf91e21dab4ce0e8fa471e9e90df3ddc04ac4f28723079bd487ea51d05f9"}
Oct 30 21:53:59.784106 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:59.784004    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/af83b188-09cb-4621-a9da-be33856d19f0-original-pull-secret\") pod \"global-pull-secret-syncer-q7fwz\" (UID: \"af83b188-09cb-4621-a9da-be33856d19f0\") " pod="kube-system/global-pull-secret-syncer-q7fwz"
Oct 30 21:53:59.784106 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:59.784057    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/532658b6-c86e-4feb-89cb-14a91316a7a0-metrics-certs\") pod \"network-metrics-daemon-n8xrh\" (UID: \"532658b6-c86e-4feb-89cb-14a91316a7a0\") " pod="openshift-multus/network-metrics-daemon-n8xrh"
Oct 30 21:53:59.784295 ip-10-0-143-246 kubenswrapper[2577]: E1030 21:53:59.784147    2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 30 21:53:59.784295 ip-10-0-143-246 kubenswrapper[2577]: E1030 21:53:59.784165    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 21:53:59.784295 ip-10-0-143-246 kubenswrapper[2577]: E1030 21:53:59.784216    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/af83b188-09cb-4621-a9da-be33856d19f0-original-pull-secret podName:af83b188-09cb-4621-a9da-be33856d19f0 nodeName:}" failed. No retries permitted until 2026-10-30 21:54:15.784202934 +0000 UTC m=+33.346615755 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/af83b188-09cb-4621-a9da-be33856d19f0-original-pull-secret") pod "global-pull-secret-syncer-q7fwz" (UID: "af83b188-09cb-4621-a9da-be33856d19f0") : object "kube-system"/"original-pull-secret" not registered
Oct 30 21:53:59.784295 ip-10-0-143-246 kubenswrapper[2577]: E1030 21:53:59.784230    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/532658b6-c86e-4feb-89cb-14a91316a7a0-metrics-certs podName:532658b6-c86e-4feb-89cb-14a91316a7a0 nodeName:}" failed. No retries permitted until 2026-10-30 21:54:15.784224631 +0000 UTC m=+33.346637452 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/532658b6-c86e-4feb-89cb-14a91316a7a0-metrics-certs") pod "network-metrics-daemon-n8xrh" (UID: "532658b6-c86e-4feb-89cb-14a91316a7a0") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 21:53:59.885246 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:53:59.885202    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ljsqq\" (UniqueName: \"kubernetes.io/projected/32ae0d42-4b76-47b7-b9d6-26b09f00ba5d-kube-api-access-ljsqq\") pod \"network-check-target-fvbz8\" (UID: \"32ae0d42-4b76-47b7-b9d6-26b09f00ba5d\") " pod="openshift-network-diagnostics/network-check-target-fvbz8"
Oct 30 21:53:59.885426 ip-10-0-143-246 kubenswrapper[2577]: E1030 21:53:59.885327    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 30 21:53:59.885426 ip-10-0-143-246 kubenswrapper[2577]: E1030 21:53:59.885341    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 30 21:53:59.885426 ip-10-0-143-246 kubenswrapper[2577]: E1030 21:53:59.885354    2577 projected.go:196] Error preparing data for projected volume kube-api-access-ljsqq for pod openshift-network-diagnostics/network-check-target-fvbz8: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 21:53:59.885426 ip-10-0-143-246 kubenswrapper[2577]: E1030 21:53:59.885399    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/32ae0d42-4b76-47b7-b9d6-26b09f00ba5d-kube-api-access-ljsqq podName:32ae0d42-4b76-47b7-b9d6-26b09f00ba5d nodeName:}" failed. No retries permitted until 2026-10-30 21:54:15.885388045 +0000 UTC m=+33.447800866 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-ljsqq" (UniqueName: "kubernetes.io/projected/32ae0d42-4b76-47b7-b9d6-26b09f00ba5d-kube-api-access-ljsqq") pod "network-check-target-fvbz8" (UID: "32ae0d42-4b76-47b7-b9d6-26b09f00ba5d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 21:54:00.065060 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:00.064972    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-q7fwz"
Oct 30 21:54:00.065060 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:00.065049    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fvbz8"
Oct 30 21:54:00.065246 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:00.065064    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-n8xrh"
Oct 30 21:54:00.065246 ip-10-0-143-246 kubenswrapper[2577]: E1030 21:54:00.065149    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fvbz8" podUID="32ae0d42-4b76-47b7-b9d6-26b09f00ba5d"
Oct 30 21:54:00.065246 ip-10-0-143-246 kubenswrapper[2577]: E1030 21:54:00.065197    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-q7fwz" podUID="af83b188-09cb-4621-a9da-be33856d19f0"
Oct 30 21:54:00.065389 ip-10-0-143-246 kubenswrapper[2577]: E1030 21:54:00.065262    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-n8xrh" podUID="532658b6-c86e-4feb-89cb-14a91316a7a0"
Oct 30 21:54:00.354896 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:00.354810    2577 generic.go:356] "Generic (PLEG): container finished" podID="3e60fceb-8682-45cc-a472-f3d7017c3395" containerID="524aa4acadbe157d10a1ca4405db94dbf41a555bf00c84ccec93c047e8e91d67" exitCode=0
Oct 30 21:54:00.355366 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:00.354901    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-c9tnl" event={"ID":"3e60fceb-8682-45cc-a472-f3d7017c3395","Type":"ContainerDied","Data":"524aa4acadbe157d10a1ca4405db94dbf41a555bf00c84ccec93c047e8e91d67"}
Oct 30 21:54:00.357787 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:00.357763    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mn9vl" event={"ID":"446beacf-70a0-4d47-96ea-bbe1ca4a5f28","Type":"ContainerStarted","Data":"dbda186d68355a13b27a756996ef61cfd328f7e2859efb1b151864a4420ef69d"}
Oct 30 21:54:00.357886 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:00.357822    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mn9vl" event={"ID":"446beacf-70a0-4d47-96ea-bbe1ca4a5f28","Type":"ContainerStarted","Data":"29fdb0c24947c5923c1024d2f039de4abc693d985b6b5b83225c2273314681e5"}
Oct 30 21:54:00.357886 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:00.357851    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mn9vl" event={"ID":"446beacf-70a0-4d47-96ea-bbe1ca4a5f28","Type":"ContainerStarted","Data":"9956bb9c88f990c9ecf28c1a696cc13c7585f025ad03819aea66136a1027debf"}
Oct 30 21:54:01.362614 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:01.362574    2577 generic.go:356] "Generic (PLEG): container finished" podID="3e60fceb-8682-45cc-a472-f3d7017c3395" containerID="3fb85fa86680ca79432d7d50189dc1d7161885943aac070e88bd75560ad8777a" exitCode=0
Oct 30 21:54:01.363096 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:01.362637    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-c9tnl" event={"ID":"3e60fceb-8682-45cc-a472-f3d7017c3395","Type":"ContainerDied","Data":"3fb85fa86680ca79432d7d50189dc1d7161885943aac070e88bd75560ad8777a"}
Oct 30 21:54:02.065079 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:02.064964    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-q7fwz"
Oct 30 21:54:02.065238 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:02.064964    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-n8xrh"
Oct 30 21:54:02.065238 ip-10-0-143-246 kubenswrapper[2577]: E1030 21:54:02.065112    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-q7fwz" podUID="af83b188-09cb-4621-a9da-be33856d19f0"
Oct 30 21:54:02.065238 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:02.064976    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fvbz8"
Oct 30 21:54:02.065238 ip-10-0-143-246 kubenswrapper[2577]: E1030 21:54:02.065179    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-n8xrh" podUID="532658b6-c86e-4feb-89cb-14a91316a7a0"
Oct 30 21:54:02.065356 ip-10-0-143-246 kubenswrapper[2577]: E1030 21:54:02.065238    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fvbz8" podUID="32ae0d42-4b76-47b7-b9d6-26b09f00ba5d"
Oct 30 21:54:02.368555 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:02.368469    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-c9tnl" event={"ID":"3e60fceb-8682-45cc-a472-f3d7017c3395","Type":"ContainerStarted","Data":"cd65735c729231cd1eb78c89ac6daf18fa37faab01ccbc40ed553e3d81d51202"}
Oct 30 21:54:02.371585 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:02.371547    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mn9vl" event={"ID":"446beacf-70a0-4d47-96ea-bbe1ca4a5f28","Type":"ContainerStarted","Data":"45d3fc489cbe48e9e9faf8eb08780e04c0d47e3460396d2f60cf0e2432d591f4"}
Oct 30 21:54:04.064624 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:04.064343    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-q7fwz"
Oct 30 21:54:04.064624 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:04.064358    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fvbz8"
Oct 30 21:54:04.065247 ip-10-0-143-246 kubenswrapper[2577]: E1030 21:54:04.064639    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-q7fwz" podUID="af83b188-09cb-4621-a9da-be33856d19f0"
Oct 30 21:54:04.065247 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:04.064387    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-n8xrh"
Oct 30 21:54:04.065247 ip-10-0-143-246 kubenswrapper[2577]: E1030 21:54:04.064696    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fvbz8" podUID="32ae0d42-4b76-47b7-b9d6-26b09f00ba5d"
Oct 30 21:54:04.065247 ip-10-0-143-246 kubenswrapper[2577]: E1030 21:54:04.064793    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-n8xrh" podUID="532658b6-c86e-4feb-89cb-14a91316a7a0"
Oct 30 21:54:04.379718 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:04.379636    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mn9vl" event={"ID":"446beacf-70a0-4d47-96ea-bbe1ca4a5f28","Type":"ContainerStarted","Data":"077689f30e7d6857974c53d725db8d7e9e8176621d9cc22f5ebb8aea2dd87b52"}
Oct 30 21:54:04.379949 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:04.379929    2577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-mn9vl"
Oct 30 21:54:04.380018 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:04.380010    2577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-mn9vl"
Oct 30 21:54:04.396068 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:04.396041    2577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-mn9vl"
Oct 30 21:54:04.475719 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:04.475663    2577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-c9tnl" podStartSLOduration=11.115914872 podStartE2EDuration="21.475648619s" totalImagesPullingTime="10.359733747s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 21:53:43 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:54:02.449309475 +0000 UTC m=+20.011722337" watchObservedRunningTime="2026-10-30 21:54:04.475648619 +0000 UTC m=+22.038061532"
Oct 30 21:54:04.476169 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:04.476143    2577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-mn9vl" podStartSLOduration=6.950035672 podStartE2EDuration="21.476134179s" totalImagesPullingTime="14.526098507s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 21:53:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:54:04.462422304 +0000 UTC m=+22.024835159" watchObservedRunningTime="2026-10-30 21:54:04.476134179 +0000 UTC m=+22.038547067"
Oct 30 21:54:05.381871 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:05.381840    2577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-mn9vl"
Oct 30 21:54:05.396314 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:05.396288    2577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-mn9vl"
Oct 30 21:54:06.064773 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:06.064739    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-q7fwz"
Oct 30 21:54:06.064962 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:06.064823    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-n8xrh"
Oct 30 21:54:06.064962 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:06.064843    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fvbz8"
Oct 30 21:54:06.064962 ip-10-0-143-246 kubenswrapper[2577]: E1030 21:54:06.064936    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-q7fwz" podUID="af83b188-09cb-4621-a9da-be33856d19f0"
Oct 30 21:54:06.065153 ip-10-0-143-246 kubenswrapper[2577]: E1030 21:54:06.065097    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-n8xrh" podUID="532658b6-c86e-4feb-89cb-14a91316a7a0"
Oct 30 21:54:06.065209 ip-10-0-143-246 kubenswrapper[2577]: E1030 21:54:06.065188    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fvbz8" podUID="32ae0d42-4b76-47b7-b9d6-26b09f00ba5d"
Oct 30 21:54:08.064361 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:08.064327    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-n8xrh"
Oct 30 21:54:08.064877 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:08.064369    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fvbz8"
Oct 30 21:54:08.064877 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:08.064369    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-q7fwz"
Oct 30 21:54:08.064877 ip-10-0-143-246 kubenswrapper[2577]: E1030 21:54:08.064435    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-n8xrh" podUID="532658b6-c86e-4feb-89cb-14a91316a7a0"
Oct 30 21:54:08.064877 ip-10-0-143-246 kubenswrapper[2577]: E1030 21:54:08.064538    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-q7fwz" podUID="af83b188-09cb-4621-a9da-be33856d19f0"
Oct 30 21:54:08.064877 ip-10-0-143-246 kubenswrapper[2577]: E1030 21:54:08.064615    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fvbz8" podUID="32ae0d42-4b76-47b7-b9d6-26b09f00ba5d"
Oct 30 21:54:10.064432 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:10.064391    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-n8xrh"
Oct 30 21:54:10.064432 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:10.064432    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-q7fwz"
Oct 30 21:54:10.064906 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:10.064435    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fvbz8"
Oct 30 21:54:10.064906 ip-10-0-143-246 kubenswrapper[2577]: E1030 21:54:10.064506    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-n8xrh" podUID="532658b6-c86e-4feb-89cb-14a91316a7a0"
Oct 30 21:54:10.064906 ip-10-0-143-246 kubenswrapper[2577]: E1030 21:54:10.064590    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fvbz8" podUID="32ae0d42-4b76-47b7-b9d6-26b09f00ba5d"
Oct 30 21:54:10.064906 ip-10-0-143-246 kubenswrapper[2577]: E1030 21:54:10.064655    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-q7fwz" podUID="af83b188-09cb-4621-a9da-be33856d19f0"
Oct 30 21:54:12.064825 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:12.064749    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-n8xrh"
Oct 30 21:54:12.065278 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:12.064758    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-q7fwz"
Oct 30 21:54:12.065278 ip-10-0-143-246 kubenswrapper[2577]: E1030 21:54:12.064917    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-n8xrh" podUID="532658b6-c86e-4feb-89cb-14a91316a7a0"
Oct 30 21:54:12.065278 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:12.064771    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fvbz8"
Oct 30 21:54:12.065278 ip-10-0-143-246 kubenswrapper[2577]: E1030 21:54:12.064979    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-q7fwz" podUID="af83b188-09cb-4621-a9da-be33856d19f0"
Oct 30 21:54:12.065278 ip-10-0-143-246 kubenswrapper[2577]: E1030 21:54:12.065064    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fvbz8" podUID="32ae0d42-4b76-47b7-b9d6-26b09f00ba5d"
Oct 30 21:54:14.065117 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:14.065081    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-n8xrh"
Oct 30 21:54:14.065569 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:14.065081    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fvbz8"
Oct 30 21:54:14.065569 ip-10-0-143-246 kubenswrapper[2577]: E1030 21:54:14.065197    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-n8xrh" podUID="532658b6-c86e-4feb-89cb-14a91316a7a0"
Oct 30 21:54:14.065569 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:14.065081    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-q7fwz"
Oct 30 21:54:14.065569 ip-10-0-143-246 kubenswrapper[2577]: E1030 21:54:14.065259    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fvbz8" podUID="32ae0d42-4b76-47b7-b9d6-26b09f00ba5d"
Oct 30 21:54:14.065569 ip-10-0-143-246 kubenswrapper[2577]: E1030 21:54:14.065322    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-q7fwz" podUID="af83b188-09cb-4621-a9da-be33856d19f0"
Oct 30 21:54:14.296804 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:14.296764    2577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-thksb"
Oct 30 21:54:14.297029 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:14.296918    2577 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Oct 30 21:54:14.297516 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:14.297496    2577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-thksb"
Oct 30 21:54:15.791250 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:15.791209    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/af83b188-09cb-4621-a9da-be33856d19f0-original-pull-secret\") pod \"global-pull-secret-syncer-q7fwz\" (UID: \"af83b188-09cb-4621-a9da-be33856d19f0\") " pod="kube-system/global-pull-secret-syncer-q7fwz"
Oct 30 21:54:15.791250 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:15.791253    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/532658b6-c86e-4feb-89cb-14a91316a7a0-metrics-certs\") pod \"network-metrics-daemon-n8xrh\" (UID: \"532658b6-c86e-4feb-89cb-14a91316a7a0\") " pod="openshift-multus/network-metrics-daemon-n8xrh"
Oct 30 21:54:15.791836 ip-10-0-143-246 kubenswrapper[2577]: E1030 21:54:15.791399    2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 30 21:54:15.791836 ip-10-0-143-246 kubenswrapper[2577]: E1030 21:54:15.791469    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 21:54:15.791836 ip-10-0-143-246 kubenswrapper[2577]: E1030 21:54:15.791484    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/af83b188-09cb-4621-a9da-be33856d19f0-original-pull-secret podName:af83b188-09cb-4621-a9da-be33856d19f0 nodeName:}" failed. No retries permitted until 2026-10-30 21:54:47.791464205 +0000 UTC m=+65.353877028 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/af83b188-09cb-4621-a9da-be33856d19f0-original-pull-secret") pod "global-pull-secret-syncer-q7fwz" (UID: "af83b188-09cb-4621-a9da-be33856d19f0") : object "kube-system"/"original-pull-secret" not registered
Oct 30 21:54:15.791836 ip-10-0-143-246 kubenswrapper[2577]: E1030 21:54:15.791530    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/532658b6-c86e-4feb-89cb-14a91316a7a0-metrics-certs podName:532658b6-c86e-4feb-89cb-14a91316a7a0 nodeName:}" failed. No retries permitted until 2026-10-30 21:54:47.791519142 +0000 UTC m=+65.353931962 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/532658b6-c86e-4feb-89cb-14a91316a7a0-metrics-certs") pod "network-metrics-daemon-n8xrh" (UID: "532658b6-c86e-4feb-89cb-14a91316a7a0") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 21:54:15.892711 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:15.892674    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ljsqq\" (UniqueName: \"kubernetes.io/projected/32ae0d42-4b76-47b7-b9d6-26b09f00ba5d-kube-api-access-ljsqq\") pod \"network-check-target-fvbz8\" (UID: \"32ae0d42-4b76-47b7-b9d6-26b09f00ba5d\") " pod="openshift-network-diagnostics/network-check-target-fvbz8"
Oct 30 21:54:15.892881 ip-10-0-143-246 kubenswrapper[2577]: E1030 21:54:15.892844    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 30 21:54:15.892881 ip-10-0-143-246 kubenswrapper[2577]: E1030 21:54:15.892870    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 30 21:54:15.892881 ip-10-0-143-246 kubenswrapper[2577]: E1030 21:54:15.892880    2577 projected.go:196] Error preparing data for projected volume kube-api-access-ljsqq for pod openshift-network-diagnostics/network-check-target-fvbz8: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 21:54:15.892981 ip-10-0-143-246 kubenswrapper[2577]: E1030 21:54:15.892934    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/32ae0d42-4b76-47b7-b9d6-26b09f00ba5d-kube-api-access-ljsqq podName:32ae0d42-4b76-47b7-b9d6-26b09f00ba5d nodeName:}" failed. No retries permitted until 2026-10-30 21:54:47.892918145 +0000 UTC m=+65.455330985 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-ljsqq" (UniqueName: "kubernetes.io/projected/32ae0d42-4b76-47b7-b9d6-26b09f00ba5d-kube-api-access-ljsqq") pod "network-check-target-fvbz8" (UID: "32ae0d42-4b76-47b7-b9d6-26b09f00ba5d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 21:54:16.064485 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:16.064394    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-q7fwz"
Oct 30 21:54:16.064485 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:16.064454    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-n8xrh"
Oct 30 21:54:16.064704 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:16.064408    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fvbz8"
Oct 30 21:54:16.064704 ip-10-0-143-246 kubenswrapper[2577]: E1030 21:54:16.064531    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-q7fwz" podUID="af83b188-09cb-4621-a9da-be33856d19f0"
Oct 30 21:54:16.064704 ip-10-0-143-246 kubenswrapper[2577]: E1030 21:54:16.064616    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fvbz8" podUID="32ae0d42-4b76-47b7-b9d6-26b09f00ba5d"
Oct 30 21:54:16.064704 ip-10-0-143-246 kubenswrapper[2577]: E1030 21:54:16.064678    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-n8xrh" podUID="532658b6-c86e-4feb-89cb-14a91316a7a0"
Oct 30 21:54:16.954100 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:16.954064    2577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-fvbz8"]
Oct 30 21:54:16.954563 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:16.954193    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fvbz8"
Oct 30 21:54:16.954563 ip-10-0-143-246 kubenswrapper[2577]: E1030 21:54:16.954303    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fvbz8" podUID="32ae0d42-4b76-47b7-b9d6-26b09f00ba5d"
Oct 30 21:54:16.957567 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:16.957535    2577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-n8xrh"]
Oct 30 21:54:16.957734 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:16.957654    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-n8xrh"
Oct 30 21:54:16.957798 ip-10-0-143-246 kubenswrapper[2577]: E1030 21:54:16.957762    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-n8xrh" podUID="532658b6-c86e-4feb-89cb-14a91316a7a0"
Oct 30 21:54:16.958770 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:16.958749    2577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-q7fwz"]
Oct 30 21:54:16.958884 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:16.958833    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-q7fwz"
Oct 30 21:54:16.958946 ip-10-0-143-246 kubenswrapper[2577]: E1030 21:54:16.958921    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-q7fwz" podUID="af83b188-09cb-4621-a9da-be33856d19f0"
Oct 30 21:54:18.308417 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:18.308192    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-wwb9f_fc4a67d9-8c7c-4f71-9a8a-2bba09329bc6/dns-node-resolver/0.log"
Oct 30 21:54:19.064592 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:19.064552    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-q7fwz"
Oct 30 21:54:19.064592 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:19.064588    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-n8xrh"
Oct 30 21:54:19.064859 ip-10-0-143-246 kubenswrapper[2577]: E1030 21:54:19.064662    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-q7fwz" podUID="af83b188-09cb-4621-a9da-be33856d19f0"
Oct 30 21:54:19.064859 ip-10-0-143-246 kubenswrapper[2577]: E1030 21:54:19.064776    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-n8xrh" podUID="532658b6-c86e-4feb-89cb-14a91316a7a0"
Oct 30 21:54:19.064979 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:19.064862    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fvbz8"
Oct 30 21:54:19.064979 ip-10-0-143-246 kubenswrapper[2577]: E1030 21:54:19.064911    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fvbz8" podUID="32ae0d42-4b76-47b7-b9d6-26b09f00ba5d"
Oct 30 21:54:19.508882 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:19.508812    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-qzgkj_fe472529-1294-46a1-a666-dc00d5d37eca/node-ca/0.log"
Oct 30 21:54:21.064661 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:21.064623    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fvbz8"
Oct 30 21:54:21.064661 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:21.064665    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-q7fwz"
Oct 30 21:54:21.065184 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:21.064633    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-n8xrh"
Oct 30 21:54:21.065184 ip-10-0-143-246 kubenswrapper[2577]: E1030 21:54:21.064755    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fvbz8" podUID="32ae0d42-4b76-47b7-b9d6-26b09f00ba5d"
Oct 30 21:54:21.065184 ip-10-0-143-246 kubenswrapper[2577]: E1030 21:54:21.064814    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-n8xrh" podUID="532658b6-c86e-4feb-89cb-14a91316a7a0"
Oct 30 21:54:21.065184 ip-10-0-143-246 kubenswrapper[2577]: E1030 21:54:21.064909    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-q7fwz" podUID="af83b188-09cb-4621-a9da-be33856d19f0"
Oct 30 21:54:21.215757 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:21.215674    2577 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-143-246.ec2.internal" event="NodeReady"
Oct 30 21:54:21.215968 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:21.215825    2577 kubelet_node_status.go:477] "Fast updating node status as it just became ready"
Oct 30 21:54:21.292519 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:21.292484    2577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-6jppj"]
Oct 30 21:54:21.311540 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:21.311504    2577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-fqmbc"]
Oct 30 21:54:21.311698 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:21.311619    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-6jppj"
Oct 30 21:54:21.323375 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:21.323352    2577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-6jppj"]
Oct 30 21:54:21.323529 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:21.323469    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-fqmbc"
Oct 30 21:54:21.325109 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:21.325085    2577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-fqmbc"]
Oct 30 21:54:21.328920 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:21.328898    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Oct 30 21:54:21.329128 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:21.328904    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Oct 30 21:54:21.329128 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:21.329069    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Oct 30 21:54:21.329558 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:21.329540    2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Oct 30 21:54:21.329638 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:21.329578    2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-v8mh7\""
Oct 30 21:54:21.339522 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:21.339494    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Oct 30 21:54:21.343682 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:21.343664    2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-bv47g\""
Oct 30 21:54:21.345621 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:21.345605    2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Oct 30 21:54:21.410244 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:21.410212    2577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-fl5mx"]
Oct 30 21:54:21.415838 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:21.415814    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-fl5mx"
Oct 30 21:54:21.430866 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:21.430835    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6192d2be-bba7-4099-b54c-e7e9b8d75389-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6jppj\" (UID: \"6192d2be-bba7-4099-b54c-e7e9b8d75389\") " pod="openshift-insights/insights-runtime-extractor-6jppj"
Oct 30 21:54:21.431038 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:21.430872    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/209b4e58-e6c2-48f1-8589-f7b6be79720b-metrics-tls\") pod \"dns-default-fqmbc\" (UID: \"209b4e58-e6c2-48f1-8589-f7b6be79720b\") " pod="openshift-dns/dns-default-fqmbc"
Oct 30 21:54:21.431038 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:21.430953    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/209b4e58-e6c2-48f1-8589-f7b6be79720b-tmp-dir\") pod \"dns-default-fqmbc\" (UID: \"209b4e58-e6c2-48f1-8589-f7b6be79720b\") " pod="openshift-dns/dns-default-fqmbc"
Oct 30 21:54:21.431038 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:21.431005    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/6192d2be-bba7-4099-b54c-e7e9b8d75389-crio-socket\") pod \"insights-runtime-extractor-6jppj\" (UID: \"6192d2be-bba7-4099-b54c-e7e9b8d75389\") " pod="openshift-insights/insights-runtime-extractor-6jppj"
Oct 30 21:54:21.431206 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:21.431047    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/209b4e58-e6c2-48f1-8589-f7b6be79720b-config-volume\") pod \"dns-default-fqmbc\" (UID: \"209b4e58-e6c2-48f1-8589-f7b6be79720b\") " pod="openshift-dns/dns-default-fqmbc"
Oct 30 21:54:21.431206 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:21.431157    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/6192d2be-bba7-4099-b54c-e7e9b8d75389-data-volume\") pod \"insights-runtime-extractor-6jppj\" (UID: \"6192d2be-bba7-4099-b54c-e7e9b8d75389\") " pod="openshift-insights/insights-runtime-extractor-6jppj"
Oct 30 21:54:21.431206 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:21.431194    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mrtmr\" (UniqueName: \"kubernetes.io/projected/209b4e58-e6c2-48f1-8589-f7b6be79720b-kube-api-access-mrtmr\") pod \"dns-default-fqmbc\" (UID: \"209b4e58-e6c2-48f1-8589-f7b6be79720b\") " pod="openshift-dns/dns-default-fqmbc"
Oct 30 21:54:21.431309 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:21.431255    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jw4wk\" (UniqueName: \"kubernetes.io/projected/6192d2be-bba7-4099-b54c-e7e9b8d75389-kube-api-access-jw4wk\") pod \"insights-runtime-extractor-6jppj\" (UID: \"6192d2be-bba7-4099-b54c-e7e9b8d75389\") " pod="openshift-insights/insights-runtime-extractor-6jppj"
Oct 30 21:54:21.431309 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:21.431281    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6192d2be-bba7-4099-b54c-e7e9b8d75389-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-6jppj\" (UID: \"6192d2be-bba7-4099-b54c-e7e9b8d75389\") " pod="openshift-insights/insights-runtime-extractor-6jppj"
Oct 30 21:54:21.435425 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:21.435407    2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Oct 30 21:54:21.438844 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:21.438821    2577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-fl5mx"]
Oct 30 21:54:21.439106 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:21.439085    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Oct 30 21:54:21.439106 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:21.439099    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Oct 30 21:54:21.439251 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:21.439146    2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"ingress-canary-dockercfg-5mpht\""
Oct 30 21:54:21.532623 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:21.532526    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/6192d2be-bba7-4099-b54c-e7e9b8d75389-crio-socket\") pod \"insights-runtime-extractor-6jppj\" (UID: \"6192d2be-bba7-4099-b54c-e7e9b8d75389\") " pod="openshift-insights/insights-runtime-extractor-6jppj"
Oct 30 21:54:21.532623 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:21.532572    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/209b4e58-e6c2-48f1-8589-f7b6be79720b-config-volume\") pod \"dns-default-fqmbc\" (UID: \"209b4e58-e6c2-48f1-8589-f7b6be79720b\") " pod="openshift-dns/dns-default-fqmbc"
Oct 30 21:54:21.532623 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:21.532599    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9f2kg\" (UniqueName: \"kubernetes.io/projected/4621048c-230e-441f-85fc-423e4ba525ec-kube-api-access-9f2kg\") pod \"ingress-canary-fl5mx\" (UID: \"4621048c-230e-441f-85fc-423e4ba525ec\") " pod="openshift-ingress-canary/ingress-canary-fl5mx"
Oct 30 21:54:21.532921 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:21.532631    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/6192d2be-bba7-4099-b54c-e7e9b8d75389-data-volume\") pod \"insights-runtime-extractor-6jppj\" (UID: \"6192d2be-bba7-4099-b54c-e7e9b8d75389\") " pod="openshift-insights/insights-runtime-extractor-6jppj"
Oct 30 21:54:21.532921 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:21.532648    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4621048c-230e-441f-85fc-423e4ba525ec-cert\") pod \"ingress-canary-fl5mx\" (UID: \"4621048c-230e-441f-85fc-423e4ba525ec\") " pod="openshift-ingress-canary/ingress-canary-fl5mx"
Oct 30 21:54:21.532921 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:21.532673    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mrtmr\" (UniqueName: \"kubernetes.io/projected/209b4e58-e6c2-48f1-8589-f7b6be79720b-kube-api-access-mrtmr\") pod \"dns-default-fqmbc\" (UID: \"209b4e58-e6c2-48f1-8589-f7b6be79720b\") " pod="openshift-dns/dns-default-fqmbc"
Oct 30 21:54:21.532921 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:21.532695    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jw4wk\" (UniqueName: \"kubernetes.io/projected/6192d2be-bba7-4099-b54c-e7e9b8d75389-kube-api-access-jw4wk\") pod \"insights-runtime-extractor-6jppj\" (UID: \"6192d2be-bba7-4099-b54c-e7e9b8d75389\") " pod="openshift-insights/insights-runtime-extractor-6jppj"
Oct 30 21:54:21.532921 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:21.532764    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6192d2be-bba7-4099-b54c-e7e9b8d75389-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-6jppj\" (UID: \"6192d2be-bba7-4099-b54c-e7e9b8d75389\") " pod="openshift-insights/insights-runtime-extractor-6jppj"
Oct 30 21:54:21.532921 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:21.532839    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/6192d2be-bba7-4099-b54c-e7e9b8d75389-crio-socket\") pod \"insights-runtime-extractor-6jppj\" (UID: \"6192d2be-bba7-4099-b54c-e7e9b8d75389\") " pod="openshift-insights/insights-runtime-extractor-6jppj"
Oct 30 21:54:21.532921 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:21.532851    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6192d2be-bba7-4099-b54c-e7e9b8d75389-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6jppj\" (UID: \"6192d2be-bba7-4099-b54c-e7e9b8d75389\") " pod="openshift-insights/insights-runtime-extractor-6jppj"
Oct 30 21:54:21.532921 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:21.532897    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/209b4e58-e6c2-48f1-8589-f7b6be79720b-metrics-tls\") pod \"dns-default-fqmbc\" (UID: \"209b4e58-e6c2-48f1-8589-f7b6be79720b\") " pod="openshift-dns/dns-default-fqmbc"
Oct 30 21:54:21.533228 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:21.532937    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/209b4e58-e6c2-48f1-8589-f7b6be79720b-tmp-dir\") pod \"dns-default-fqmbc\" (UID: \"209b4e58-e6c2-48f1-8589-f7b6be79720b\") " pod="openshift-dns/dns-default-fqmbc"
Oct 30 21:54:21.533228 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:21.533021    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/6192d2be-bba7-4099-b54c-e7e9b8d75389-data-volume\") pod \"insights-runtime-extractor-6jppj\" (UID: \"6192d2be-bba7-4099-b54c-e7e9b8d75389\") " pod="openshift-insights/insights-runtime-extractor-6jppj"
Oct 30 21:54:21.533331 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:21.533313    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/209b4e58-e6c2-48f1-8589-f7b6be79720b-tmp-dir\") pod \"dns-default-fqmbc\" (UID: \"209b4e58-e6c2-48f1-8589-f7b6be79720b\") " pod="openshift-dns/dns-default-fqmbc"
Oct 30 21:54:21.533392 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:21.533376    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6192d2be-bba7-4099-b54c-e7e9b8d75389-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-6jppj\" (UID: \"6192d2be-bba7-4099-b54c-e7e9b8d75389\") " pod="openshift-insights/insights-runtime-extractor-6jppj"
Oct 30 21:54:21.533714 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:21.533696    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/209b4e58-e6c2-48f1-8589-f7b6be79720b-config-volume\") pod \"dns-default-fqmbc\" (UID: \"209b4e58-e6c2-48f1-8589-f7b6be79720b\") " pod="openshift-dns/dns-default-fqmbc"
Oct 30 21:54:21.536804 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:21.536782    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6192d2be-bba7-4099-b54c-e7e9b8d75389-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6jppj\" (UID: \"6192d2be-bba7-4099-b54c-e7e9b8d75389\") " pod="openshift-insights/insights-runtime-extractor-6jppj"
Oct 30 21:54:21.536924 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:21.536784    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/209b4e58-e6c2-48f1-8589-f7b6be79720b-metrics-tls\") pod \"dns-default-fqmbc\" (UID: \"209b4e58-e6c2-48f1-8589-f7b6be79720b\") " pod="openshift-dns/dns-default-fqmbc"
Oct 30 21:54:21.554853 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:21.554820    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jw4wk\" (UniqueName: \"kubernetes.io/projected/6192d2be-bba7-4099-b54c-e7e9b8d75389-kube-api-access-jw4wk\") pod \"insights-runtime-extractor-6jppj\" (UID: \"6192d2be-bba7-4099-b54c-e7e9b8d75389\") " pod="openshift-insights/insights-runtime-extractor-6jppj"
Oct 30 21:54:21.555541 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:21.555525    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mrtmr\" (UniqueName: \"kubernetes.io/projected/209b4e58-e6c2-48f1-8589-f7b6be79720b-kube-api-access-mrtmr\") pod \"dns-default-fqmbc\" (UID: \"209b4e58-e6c2-48f1-8589-f7b6be79720b\") " pod="openshift-dns/dns-default-fqmbc"
Oct 30 21:54:21.633406 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:21.633367    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9f2kg\" (UniqueName: \"kubernetes.io/projected/4621048c-230e-441f-85fc-423e4ba525ec-kube-api-access-9f2kg\") pod \"ingress-canary-fl5mx\" (UID: \"4621048c-230e-441f-85fc-423e4ba525ec\") " pod="openshift-ingress-canary/ingress-canary-fl5mx"
Oct 30 21:54:21.633406 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:21.633410    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4621048c-230e-441f-85fc-423e4ba525ec-cert\") pod \"ingress-canary-fl5mx\" (UID: \"4621048c-230e-441f-85fc-423e4ba525ec\") " pod="openshift-ingress-canary/ingress-canary-fl5mx"
Oct 30 21:54:21.635853 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:21.635831    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4621048c-230e-441f-85fc-423e4ba525ec-cert\") pod \"ingress-canary-fl5mx\" (UID: \"4621048c-230e-441f-85fc-423e4ba525ec\") " pod="openshift-ingress-canary/ingress-canary-fl5mx"
Oct 30 21:54:21.648752 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:21.648723    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-6jppj"
Oct 30 21:54:21.655530 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:21.655501    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9f2kg\" (UniqueName: \"kubernetes.io/projected/4621048c-230e-441f-85fc-423e4ba525ec-kube-api-access-9f2kg\") pod \"ingress-canary-fl5mx\" (UID: \"4621048c-230e-441f-85fc-423e4ba525ec\") " pod="openshift-ingress-canary/ingress-canary-fl5mx"
Oct 30 21:54:21.665452 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:21.665431    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-fqmbc"
Oct 30 21:54:21.729033 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:21.728978    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-fl5mx"
Oct 30 21:54:22.050247 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:22.050222    2577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-6jppj"]
Oct 30 21:54:22.051937 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:54:22.051913    2577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6192d2be_bba7_4099_b54c_e7e9b8d75389.slice/crio-8ce50405433f59d6286d2bbf2f1008372c665afdfb13349ef7546cf0b1be08a4 WatchSource:0}: Error finding container 8ce50405433f59d6286d2bbf2f1008372c665afdfb13349ef7546cf0b1be08a4: Status 404 returned error can't find the container with id 8ce50405433f59d6286d2bbf2f1008372c665afdfb13349ef7546cf0b1be08a4
Oct 30 21:54:22.056446 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:22.056423    2577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-fl5mx"]
Oct 30 21:54:22.059710 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:22.059687    2577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-fqmbc"]
Oct 30 21:54:22.060129 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:54:22.060107    2577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4621048c_230e_441f_85fc_423e4ba525ec.slice/crio-ea0896e86f6df5158e1a1ecc5b41aa8e849df46707c6cc80f76940fc2dadcd09 WatchSource:0}: Error finding container ea0896e86f6df5158e1a1ecc5b41aa8e849df46707c6cc80f76940fc2dadcd09: Status 404 returned error can't find the container with id ea0896e86f6df5158e1a1ecc5b41aa8e849df46707c6cc80f76940fc2dadcd09
Oct 30 21:54:22.060805 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:54:22.060764    2577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod209b4e58_e6c2_48f1_8589_f7b6be79720b.slice/crio-c0fd54d2f9eade40d52d73aa5fd6dfb74c9a1ed40c855e3e0773cb680ead5d10 WatchSource:0}: Error finding container c0fd54d2f9eade40d52d73aa5fd6dfb74c9a1ed40c855e3e0773cb680ead5d10: Status 404 returned error can't find the container with id c0fd54d2f9eade40d52d73aa5fd6dfb74c9a1ed40c855e3e0773cb680ead5d10
Oct 30 21:54:22.419763 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:22.419477    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-fl5mx" event={"ID":"4621048c-230e-441f-85fc-423e4ba525ec","Type":"ContainerStarted","Data":"ea0896e86f6df5158e1a1ecc5b41aa8e849df46707c6cc80f76940fc2dadcd09"}
Oct 30 21:54:22.420967 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:22.420942    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-6jppj" event={"ID":"6192d2be-bba7-4099-b54c-e7e9b8d75389","Type":"ContainerStarted","Data":"0d6f1775ef16728371e90be7bfe916bfd41a7d26316d96b47fff7b4439125a18"}
Oct 30 21:54:22.421090 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:22.421012    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-6jppj" event={"ID":"6192d2be-bba7-4099-b54c-e7e9b8d75389","Type":"ContainerStarted","Data":"8ce50405433f59d6286d2bbf2f1008372c665afdfb13349ef7546cf0b1be08a4"}
Oct 30 21:54:22.421971 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:22.421950    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-fqmbc" event={"ID":"209b4e58-e6c2-48f1-8589-f7b6be79720b","Type":"ContainerStarted","Data":"c0fd54d2f9eade40d52d73aa5fd6dfb74c9a1ed40c855e3e0773cb680ead5d10"}
Oct 30 21:54:23.065491 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:23.065456    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-n8xrh"
Oct 30 21:54:23.066233 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:23.066209    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-q7fwz"
Oct 30 21:54:23.066632 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:23.066610    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fvbz8"
Oct 30 21:54:23.169831 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:23.169778    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Oct 30 21:54:23.183654 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:23.183462    2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-check-target-dockercfg-vwffm\""
Oct 30 21:54:23.183654 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:23.183494    2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Oct 30 21:54:23.183654 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:23.183534    2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"global-pull-secret-syncer-dockercfg-cggx8\""
Oct 30 21:54:23.183654 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:23.183582    2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-2n5cw\""
Oct 30 21:54:23.183654 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:23.183626    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Oct 30 21:54:23.183965 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:23.183678    2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Oct 30 21:54:24.431768 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:24.431735    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-6jppj" event={"ID":"6192d2be-bba7-4099-b54c-e7e9b8d75389","Type":"ContainerStarted","Data":"a45c8a8f3b26ef72e081c5cf5413e54247110a2a867aeaeaf896724c06bbfe67"}
Oct 30 21:54:24.500121 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:24.500084    2577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-7bd7489957-f26w7"]
Oct 30 21:54:24.517256 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:24.517226    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-f26w7"
Oct 30 21:54:24.554613 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:24.554346    2577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-7bd7489957-f26w7"]
Oct 30 21:54:24.568588 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:24.568524    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Oct 30 21:54:24.568729 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:24.568701    2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\""
Oct 30 21:54:24.568767 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:24.568750    2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\""
Oct 30 21:54:24.568847 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:24.568831    2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-pfg22\""
Oct 30 21:54:24.568937 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:24.568922    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Oct 30 21:54:24.569107 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:24.569090    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Oct 30 21:54:24.638088 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:24.638049    2577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-df5jt"]
Oct 30 21:54:24.644404 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:24.644368    2577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-5dc8d7d89b-8727r"]
Oct 30 21:54:24.644580 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:24.644514    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-df5jt"
Oct 30 21:54:24.656610 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:24.656586    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-8727r"
Oct 30 21:54:24.657014 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:24.656911    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/484e37a0-fd1d-4fd0-a200-939b8945984b-metrics-client-ca\") pod \"openshift-state-metrics-7bd7489957-f26w7\" (UID: \"484e37a0-fd1d-4fd0-a200-939b8945984b\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-f26w7"
Oct 30 21:54:24.657014 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:24.656957    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/484e37a0-fd1d-4fd0-a200-939b8945984b-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-7bd7489957-f26w7\" (UID: \"484e37a0-fd1d-4fd0-a200-939b8945984b\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-f26w7"
Oct 30 21:54:24.657188 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:24.657028    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n87x8\" (UniqueName: \"kubernetes.io/projected/484e37a0-fd1d-4fd0-a200-939b8945984b-kube-api-access-n87x8\") pod \"openshift-state-metrics-7bd7489957-f26w7\" (UID: \"484e37a0-fd1d-4fd0-a200-939b8945984b\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-f26w7"
Oct 30 21:54:24.657188 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:24.657062    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/484e37a0-fd1d-4fd0-a200-939b8945984b-openshift-state-metrics-tls\") pod \"openshift-state-metrics-7bd7489957-f26w7\" (UID: \"484e37a0-fd1d-4fd0-a200-939b8945984b\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-f26w7"
Oct 30 21:54:24.669912 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:24.669839    2577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-5dc8d7d89b-8727r"]
Oct 30 21:54:24.755665 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:24.755634    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Oct 30 21:54:24.756036 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:24.756017    2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-5kd5n\""
Oct 30 21:54:24.756131 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:24.756112    2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\""
Oct 30 21:54:24.756186 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:24.756165    2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Oct 30 21:54:24.756237 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:24.756218    2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Oct 30 21:54:24.756300 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:24.756286    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\""
Oct 30 21:54:24.756347 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:24.756329    2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-5cssp\""
Oct 30 21:54:24.756394 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:24.756356    2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\""
Oct 30 21:54:24.757710 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:24.757682    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/484e37a0-fd1d-4fd0-a200-939b8945984b-metrics-client-ca\") pod \"openshift-state-metrics-7bd7489957-f26w7\" (UID: \"484e37a0-fd1d-4fd0-a200-939b8945984b\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-f26w7"
Oct 30 21:54:24.757813 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:24.757720    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7ddfa67d-20a0-473d-937f-25fd8e0ba80e-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-df5jt\" (UID: \"7ddfa67d-20a0-473d-937f-25fd8e0ba80e\") " pod="openshift-monitoring/node-exporter-df5jt"
Oct 30 21:54:24.757813 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:24.757746    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bw6lg\" (UniqueName: \"kubernetes.io/projected/17da628d-31f5-456c-ad37-52c884d22cf4-kube-api-access-bw6lg\") pod \"kube-state-metrics-5dc8d7d89b-8727r\" (UID: \"17da628d-31f5-456c-ad37-52c884d22cf4\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-8727r"
Oct 30 21:54:24.757900 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:24.757857    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/7ddfa67d-20a0-473d-937f-25fd8e0ba80e-node-exporter-textfile\") pod \"node-exporter-df5jt\" (UID: \"7ddfa67d-20a0-473d-937f-25fd8e0ba80e\") " pod="openshift-monitoring/node-exporter-df5jt"
Oct 30 21:54:24.757944 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:24.757900    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7ddfa67d-20a0-473d-937f-25fd8e0ba80e-metrics-client-ca\") pod \"node-exporter-df5jt\" (UID: \"7ddfa67d-20a0-473d-937f-25fd8e0ba80e\") " pod="openshift-monitoring/node-exporter-df5jt"
Oct 30 21:54:24.757944 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:24.757933    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/17da628d-31f5-456c-ad37-52c884d22cf4-kube-state-metrics-tls\") pod \"kube-state-metrics-5dc8d7d89b-8727r\" (UID: \"17da628d-31f5-456c-ad37-52c884d22cf4\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-8727r"
Oct 30 21:54:24.758040 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:24.758007    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7ddfa67d-20a0-473d-937f-25fd8e0ba80e-sys\") pod \"node-exporter-df5jt\" (UID: \"7ddfa67d-20a0-473d-937f-25fd8e0ba80e\") " pod="openshift-monitoring/node-exporter-df5jt"
Oct 30 21:54:24.758104 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:24.758074    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/484e37a0-fd1d-4fd0-a200-939b8945984b-openshift-state-metrics-tls\") pod \"openshift-state-metrics-7bd7489957-f26w7\" (UID: \"484e37a0-fd1d-4fd0-a200-939b8945984b\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-f26w7"
Oct 30 21:54:24.758559 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:24.758541    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/484e37a0-fd1d-4fd0-a200-939b8945984b-metrics-client-ca\") pod \"openshift-state-metrics-7bd7489957-f26w7\" (UID: \"484e37a0-fd1d-4fd0-a200-939b8945984b\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-f26w7"
Oct 30 21:54:24.758646 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:24.758605    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/7ddfa67d-20a0-473d-937f-25fd8e0ba80e-node-exporter-accelerators-collector-config\") pod \"node-exporter-df5jt\" (UID: \"7ddfa67d-20a0-473d-937f-25fd8e0ba80e\") " pod="openshift-monitoring/node-exporter-df5jt"
Oct 30 21:54:24.758765 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:24.758744    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/17da628d-31f5-456c-ad37-52c884d22cf4-volume-directive-shadow\") pod \"kube-state-metrics-5dc8d7d89b-8727r\" (UID: \"17da628d-31f5-456c-ad37-52c884d22cf4\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-8727r"
Oct 30 21:54:24.758839 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:24.758774    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wsnr5\" (UniqueName: \"kubernetes.io/projected/7ddfa67d-20a0-473d-937f-25fd8e0ba80e-kube-api-access-wsnr5\") pod \"node-exporter-df5jt\" (UID: \"7ddfa67d-20a0-473d-937f-25fd8e0ba80e\") " pod="openshift-monitoring/node-exporter-df5jt"
Oct 30 21:54:24.758839 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:24.758800    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/7ddfa67d-20a0-473d-937f-25fd8e0ba80e-node-exporter-tls\") pod \"node-exporter-df5jt\" (UID: \"7ddfa67d-20a0-473d-937f-25fd8e0ba80e\") " pod="openshift-monitoring/node-exporter-df5jt"
Oct 30 21:54:24.758928 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:24.758863    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n87x8\" (UniqueName: \"kubernetes.io/projected/484e37a0-fd1d-4fd0-a200-939b8945984b-kube-api-access-n87x8\") pod \"openshift-state-metrics-7bd7489957-f26w7\" (UID: \"484e37a0-fd1d-4fd0-a200-939b8945984b\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-f26w7"
Oct 30 21:54:24.758928 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:24.758914    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/17da628d-31f5-456c-ad37-52c884d22cf4-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-5dc8d7d89b-8727r\" (UID: \"17da628d-31f5-456c-ad37-52c884d22cf4\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-8727r"
Oct 30 21:54:24.759038 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:24.758945    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/17da628d-31f5-456c-ad37-52c884d22cf4-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-5dc8d7d89b-8727r\" (UID: \"17da628d-31f5-456c-ad37-52c884d22cf4\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-8727r"
Oct 30 21:54:24.759038 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:24.758974    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/7ddfa67d-20a0-473d-937f-25fd8e0ba80e-root\") pod \"node-exporter-df5jt\" (UID: \"7ddfa67d-20a0-473d-937f-25fd8e0ba80e\") " pod="openshift-monitoring/node-exporter-df5jt"
Oct 30 21:54:24.759038 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:24.759016    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/7ddfa67d-20a0-473d-937f-25fd8e0ba80e-node-exporter-wtmp\") pod \"node-exporter-df5jt\" (UID: \"7ddfa67d-20a0-473d-937f-25fd8e0ba80e\") " pod="openshift-monitoring/node-exporter-df5jt"
Oct 30 21:54:24.759182 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:24.759047    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/17da628d-31f5-456c-ad37-52c884d22cf4-metrics-client-ca\") pod \"kube-state-metrics-5dc8d7d89b-8727r\" (UID: \"17da628d-31f5-456c-ad37-52c884d22cf4\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-8727r"
Oct 30 21:54:24.759182 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:24.759086    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/484e37a0-fd1d-4fd0-a200-939b8945984b-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-7bd7489957-f26w7\" (UID: \"484e37a0-fd1d-4fd0-a200-939b8945984b\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-f26w7"
Oct 30 21:54:24.762695 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:24.762658    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/484e37a0-fd1d-4fd0-a200-939b8945984b-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-7bd7489957-f26w7\" (UID: \"484e37a0-fd1d-4fd0-a200-939b8945984b\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-f26w7"
Oct 30 21:54:24.762695 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:24.762685    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/484e37a0-fd1d-4fd0-a200-939b8945984b-openshift-state-metrics-tls\") pod \"openshift-state-metrics-7bd7489957-f26w7\" (UID: \"484e37a0-fd1d-4fd0-a200-939b8945984b\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-f26w7"
Oct 30 21:54:24.781345 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:24.781314    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n87x8\" (UniqueName: \"kubernetes.io/projected/484e37a0-fd1d-4fd0-a200-939b8945984b-kube-api-access-n87x8\") pod \"openshift-state-metrics-7bd7489957-f26w7\" (UID: \"484e37a0-fd1d-4fd0-a200-939b8945984b\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-f26w7"
Oct 30 21:54:24.860197 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:24.860106    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wsnr5\" (UniqueName: \"kubernetes.io/projected/7ddfa67d-20a0-473d-937f-25fd8e0ba80e-kube-api-access-wsnr5\") pod \"node-exporter-df5jt\" (UID: \"7ddfa67d-20a0-473d-937f-25fd8e0ba80e\") " pod="openshift-monitoring/node-exporter-df5jt"
Oct 30 21:54:24.860197 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:24.860159    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/7ddfa67d-20a0-473d-937f-25fd8e0ba80e-node-exporter-tls\") pod \"node-exporter-df5jt\" (UID: \"7ddfa67d-20a0-473d-937f-25fd8e0ba80e\") " pod="openshift-monitoring/node-exporter-df5jt"
Oct 30 21:54:24.860197 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:24.860194    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/17da628d-31f5-456c-ad37-52c884d22cf4-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-5dc8d7d89b-8727r\" (UID: \"17da628d-31f5-456c-ad37-52c884d22cf4\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-8727r"
Oct 30 21:54:24.860433 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:24.860228    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/17da628d-31f5-456c-ad37-52c884d22cf4-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-5dc8d7d89b-8727r\" (UID: \"17da628d-31f5-456c-ad37-52c884d22cf4\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-8727r"
Oct 30 21:54:24.860433 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:24.860258    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/7ddfa67d-20a0-473d-937f-25fd8e0ba80e-root\") pod \"node-exporter-df5jt\" (UID: \"7ddfa67d-20a0-473d-937f-25fd8e0ba80e\") " pod="openshift-monitoring/node-exporter-df5jt"
Oct 30 21:54:24.860433 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:24.860283    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/7ddfa67d-20a0-473d-937f-25fd8e0ba80e-node-exporter-wtmp\") pod \"node-exporter-df5jt\" (UID: \"7ddfa67d-20a0-473d-937f-25fd8e0ba80e\") " pod="openshift-monitoring/node-exporter-df5jt"
Oct 30 21:54:24.860433 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:24.860316    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/17da628d-31f5-456c-ad37-52c884d22cf4-metrics-client-ca\") pod \"kube-state-metrics-5dc8d7d89b-8727r\" (UID: \"17da628d-31f5-456c-ad37-52c884d22cf4\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-8727r"
Oct 30 21:54:24.860433 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:24.860367    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7ddfa67d-20a0-473d-937f-25fd8e0ba80e-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-df5jt\" (UID: \"7ddfa67d-20a0-473d-937f-25fd8e0ba80e\") " pod="openshift-monitoring/node-exporter-df5jt"
Oct 30 21:54:24.860433 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:24.860397    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bw6lg\" (UniqueName: \"kubernetes.io/projected/17da628d-31f5-456c-ad37-52c884d22cf4-kube-api-access-bw6lg\") pod \"kube-state-metrics-5dc8d7d89b-8727r\" (UID: \"17da628d-31f5-456c-ad37-52c884d22cf4\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-8727r"
Oct 30 21:54:24.860433 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:24.860431    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/7ddfa67d-20a0-473d-937f-25fd8e0ba80e-node-exporter-textfile\") pod \"node-exporter-df5jt\" (UID: \"7ddfa67d-20a0-473d-937f-25fd8e0ba80e\") " pod="openshift-monitoring/node-exporter-df5jt"
Oct 30 21:54:24.860837 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:24.860456    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7ddfa67d-20a0-473d-937f-25fd8e0ba80e-metrics-client-ca\") pod \"node-exporter-df5jt\" (UID: \"7ddfa67d-20a0-473d-937f-25fd8e0ba80e\") " pod="openshift-monitoring/node-exporter-df5jt"
Oct 30 21:54:24.860837 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:24.860487    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/17da628d-31f5-456c-ad37-52c884d22cf4-kube-state-metrics-tls\") pod \"kube-state-metrics-5dc8d7d89b-8727r\" (UID: \"17da628d-31f5-456c-ad37-52c884d22cf4\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-8727r"
Oct 30 21:54:24.860837 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:24.860523    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7ddfa67d-20a0-473d-937f-25fd8e0ba80e-sys\") pod \"node-exporter-df5jt\" (UID: \"7ddfa67d-20a0-473d-937f-25fd8e0ba80e\") " pod="openshift-monitoring/node-exporter-df5jt"
Oct 30 21:54:24.860837 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:24.860554    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/7ddfa67d-20a0-473d-937f-25fd8e0ba80e-node-exporter-accelerators-collector-config\") pod \"node-exporter-df5jt\" (UID: \"7ddfa67d-20a0-473d-937f-25fd8e0ba80e\") " pod="openshift-monitoring/node-exporter-df5jt"
Oct 30 21:54:24.860837 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:24.860623    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/17da628d-31f5-456c-ad37-52c884d22cf4-volume-directive-shadow\") pod \"kube-state-metrics-5dc8d7d89b-8727r\" (UID: \"17da628d-31f5-456c-ad37-52c884d22cf4\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-8727r"
Oct 30 21:54:24.861110 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:24.861034    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/17da628d-31f5-456c-ad37-52c884d22cf4-volume-directive-shadow\") pod \"kube-state-metrics-5dc8d7d89b-8727r\" (UID: \"17da628d-31f5-456c-ad37-52c884d22cf4\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-8727r"
Oct 30 21:54:24.861250 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:24.861220    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7ddfa67d-20a0-473d-937f-25fd8e0ba80e-metrics-client-ca\") pod \"node-exporter-df5jt\" (UID: \"7ddfa67d-20a0-473d-937f-25fd8e0ba80e\") " pod="openshift-monitoring/node-exporter-df5jt"
Oct 30 21:54:24.862162 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:24.861387    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/7ddfa67d-20a0-473d-937f-25fd8e0ba80e-root\") pod \"node-exporter-df5jt\" (UID: \"7ddfa67d-20a0-473d-937f-25fd8e0ba80e\") " pod="openshift-monitoring/node-exporter-df5jt"
Oct 30 21:54:24.862162 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:24.861466    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/7ddfa67d-20a0-473d-937f-25fd8e0ba80e-node-exporter-wtmp\") pod \"node-exporter-df5jt\" (UID: \"7ddfa67d-20a0-473d-937f-25fd8e0ba80e\") " pod="openshift-monitoring/node-exporter-df5jt"
Oct 30 21:54:24.862162 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:24.861799    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/7ddfa67d-20a0-473d-937f-25fd8e0ba80e-node-exporter-textfile\") pod \"node-exporter-df5jt\" (UID: \"7ddfa67d-20a0-473d-937f-25fd8e0ba80e\") " pod="openshift-monitoring/node-exporter-df5jt"
Oct 30 21:54:24.862162 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:24.861870    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7ddfa67d-20a0-473d-937f-25fd8e0ba80e-sys\") pod \"node-exporter-df5jt\" (UID: \"7ddfa67d-20a0-473d-937f-25fd8e0ba80e\") " pod="openshift-monitoring/node-exporter-df5jt"
Oct 30 21:54:24.862162 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:24.862105    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/17da628d-31f5-456c-ad37-52c884d22cf4-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-5dc8d7d89b-8727r\" (UID: \"17da628d-31f5-456c-ad37-52c884d22cf4\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-8727r"
Oct 30 21:54:24.862162 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:24.862127    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/17da628d-31f5-456c-ad37-52c884d22cf4-metrics-client-ca\") pod \"kube-state-metrics-5dc8d7d89b-8727r\" (UID: \"17da628d-31f5-456c-ad37-52c884d22cf4\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-8727r"
Oct 30 21:54:24.862481 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:24.862369    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/7ddfa67d-20a0-473d-937f-25fd8e0ba80e-node-exporter-accelerators-collector-config\") pod \"node-exporter-df5jt\" (UID: \"7ddfa67d-20a0-473d-937f-25fd8e0ba80e\") " pod="openshift-monitoring/node-exporter-df5jt"
Oct 30 21:54:24.863295 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:24.863270    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/7ddfa67d-20a0-473d-937f-25fd8e0ba80e-node-exporter-tls\") pod \"node-exporter-df5jt\" (UID: \"7ddfa67d-20a0-473d-937f-25fd8e0ba80e\") " pod="openshift-monitoring/node-exporter-df5jt"
Oct 30 21:54:24.863564 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:24.863547    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7ddfa67d-20a0-473d-937f-25fd8e0ba80e-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-df5jt\" (UID: \"7ddfa67d-20a0-473d-937f-25fd8e0ba80e\") " pod="openshift-monitoring/node-exporter-df5jt"
Oct 30 21:54:24.863683 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:24.863659    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/17da628d-31f5-456c-ad37-52c884d22cf4-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-5dc8d7d89b-8727r\" (UID: \"17da628d-31f5-456c-ad37-52c884d22cf4\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-8727r"
Oct 30 21:54:24.864346 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:24.864328    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/17da628d-31f5-456c-ad37-52c884d22cf4-kube-state-metrics-tls\") pod \"kube-state-metrics-5dc8d7d89b-8727r\" (UID: \"17da628d-31f5-456c-ad37-52c884d22cf4\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-8727r"
Oct 30 21:54:24.870187 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:24.870162    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-f26w7"
Oct 30 21:54:24.885045 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:24.885014    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bw6lg\" (UniqueName: \"kubernetes.io/projected/17da628d-31f5-456c-ad37-52c884d22cf4-kube-api-access-bw6lg\") pod \"kube-state-metrics-5dc8d7d89b-8727r\" (UID: \"17da628d-31f5-456c-ad37-52c884d22cf4\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-8727r"
Oct 30 21:54:24.886402 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:24.886374    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wsnr5\" (UniqueName: \"kubernetes.io/projected/7ddfa67d-20a0-473d-937f-25fd8e0ba80e-kube-api-access-wsnr5\") pod \"node-exporter-df5jt\" (UID: \"7ddfa67d-20a0-473d-937f-25fd8e0ba80e\") " pod="openshift-monitoring/node-exporter-df5jt"
Oct 30 21:54:24.985278 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:24.985100    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-df5jt"
Oct 30 21:54:25.006493 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:54:25.006462    2577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7ddfa67d_20a0_473d_937f_25fd8e0ba80e.slice/crio-efaaacbcd02ded84d312f70628a2cf5b042af9118274819e3ab06919565b3f46 WatchSource:0}: Error finding container efaaacbcd02ded84d312f70628a2cf5b042af9118274819e3ab06919565b3f46: Status 404 returned error can't find the container with id efaaacbcd02ded84d312f70628a2cf5b042af9118274819e3ab06919565b3f46
Oct 30 21:54:25.022739 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:25.022710    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-8727r"
Oct 30 21:54:25.187198 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:25.187175    2577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-7bd7489957-f26w7"]
Oct 30 21:54:25.188688 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:54:25.188662    2577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod484e37a0_fd1d_4fd0_a200_939b8945984b.slice/crio-81edfd255828d18cc1f4fca31e927255f649b6b1bcbba23bb65a1819dd9a9b65 WatchSource:0}: Error finding container 81edfd255828d18cc1f4fca31e927255f649b6b1bcbba23bb65a1819dd9a9b65: Status 404 returned error can't find the container with id 81edfd255828d18cc1f4fca31e927255f649b6b1bcbba23bb65a1819dd9a9b65
Oct 30 21:54:25.332501 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:25.332324    2577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-5dc8d7d89b-8727r"]
Oct 30 21:54:25.333825 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:54:25.333795    2577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod17da628d_31f5_456c_ad37_52c884d22cf4.slice/crio-e87446335be5893b70428a03a70ad83dac422290b1eff305dbe509ee7e35170d WatchSource:0}: Error finding container e87446335be5893b70428a03a70ad83dac422290b1eff305dbe509ee7e35170d: Status 404 returned error can't find the container with id e87446335be5893b70428a03a70ad83dac422290b1eff305dbe509ee7e35170d
Oct 30 21:54:25.437062 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:25.436926    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-fqmbc" event={"ID":"209b4e58-e6c2-48f1-8589-f7b6be79720b","Type":"ContainerStarted","Data":"65fd6720ca6e8a8615af93496fc8952b3ca448f19a46a296d010bf99a669dfb4"}
Oct 30 21:54:25.437062 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:25.436979    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-fqmbc" event={"ID":"209b4e58-e6c2-48f1-8589-f7b6be79720b","Type":"ContainerStarted","Data":"0150b9db296c9c39818f638622e6281502ed4dd306987e162d04094b4ed95e6d"}
Oct 30 21:54:25.437062 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:25.437050    2577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-fqmbc"
Oct 30 21:54:25.439333 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:25.439307    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-6jppj" event={"ID":"6192d2be-bba7-4099-b54c-e7e9b8d75389","Type":"ContainerStarted","Data":"bdc8cc68f6480d6a83fee4ae5217dd81a825cf37a12f2448d8eed130833cdfe1"}
Oct 30 21:54:25.440439 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:25.440420    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-8727r" event={"ID":"17da628d-31f5-456c-ad37-52c884d22cf4","Type":"ContainerStarted","Data":"e87446335be5893b70428a03a70ad83dac422290b1eff305dbe509ee7e35170d"}
Oct 30 21:54:25.441568 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:25.441546    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-df5jt" event={"ID":"7ddfa67d-20a0-473d-937f-25fd8e0ba80e","Type":"ContainerStarted","Data":"efaaacbcd02ded84d312f70628a2cf5b042af9118274819e3ab06919565b3f46"}
Oct 30 21:54:25.443258 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:25.443239    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-f26w7" event={"ID":"484e37a0-fd1d-4fd0-a200-939b8945984b","Type":"ContainerStarted","Data":"265ee44f70607a476ec2669b5621afa1c82245c29b3df8f7a36c2a8ef289f1ea"}
Oct 30 21:54:25.443363 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:25.443276    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-f26w7" event={"ID":"484e37a0-fd1d-4fd0-a200-939b8945984b","Type":"ContainerStarted","Data":"6473d43baf01214de0ff1192d82421f724743d9d0bf1d7f0b196521bcac1bea6"}
Oct 30 21:54:25.443363 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:25.443294    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-f26w7" event={"ID":"484e37a0-fd1d-4fd0-a200-939b8945984b","Type":"ContainerStarted","Data":"81edfd255828d18cc1f4fca31e927255f649b6b1bcbba23bb65a1819dd9a9b65"}
Oct 30 21:54:25.444621 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:25.444603    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-fl5mx" event={"ID":"4621048c-230e-441f-85fc-423e4ba525ec","Type":"ContainerStarted","Data":"bf1e1f9bb6c961b5c922b14f16a3f6479dffe967b23be79b6589d8ea2cee36fb"}
Oct 30 21:54:25.494328 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:25.494270    2577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-dns/dns-default-fqmbc" podStartSLOduration=2.238145226 podStartE2EDuration="4.494249938s" totalImagesPullingTime="2.256104712s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:54:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:54:25.493084569 +0000 UTC m=+43.055497441" watchObservedRunningTime="2026-10-30 21:54:25.494249938 +0000 UTC m=+43.056662784"
Oct 30 21:54:25.552535 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:25.552467    2577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-6jppj" podStartSLOduration=1.7965219559999999 podStartE2EDuration="4.552449118s" totalImagesPullingTime="2.755927162s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 21:54:21 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:54:25.552344358 +0000 UTC m=+43.114757220" watchObservedRunningTime="2026-10-30 21:54:25.552449118 +0000 UTC m=+43.114861962"
Oct 30 21:54:25.609769 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:25.609716    2577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-fl5mx" podStartSLOduration=1.781637818 podStartE2EDuration="4.609703317s" totalImagesPullingTime="2.828065499s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:54:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:54:25.608979823 +0000 UTC m=+43.171392667" watchObservedRunningTime="2026-10-30 21:54:25.609703317 +0000 UTC m=+43.172116159"
Oct 30 21:54:26.450275 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:26.450243    2577 generic.go:356] "Generic (PLEG): container finished" podID="7ddfa67d-20a0-473d-937f-25fd8e0ba80e" containerID="fd1e33dbc11f4190c42b506a8dcb51ecc84c5ff25b26b988b4b833e2b1b50381" exitCode=0
Oct 30 21:54:26.450799 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:26.450334    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-df5jt" event={"ID":"7ddfa67d-20a0-473d-937f-25fd8e0ba80e","Type":"ContainerDied","Data":"fd1e33dbc11f4190c42b506a8dcb51ecc84c5ff25b26b988b4b833e2b1b50381"}
Oct 30 21:54:26.453132 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:26.453099    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-f26w7" event={"ID":"484e37a0-fd1d-4fd0-a200-939b8945984b","Type":"ContainerStarted","Data":"d9c558d3fd894e50137c3c719d2d9c7db73bd9730c28f774f20bae40d85bf050"}
Oct 30 21:54:26.598434 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:26.598364    2577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-f26w7" podStartSLOduration=1.656851052 podStartE2EDuration="2.598341971s" totalImagesPullingTime="941.490919ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:54:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:54:26.597269547 +0000 UTC m=+44.159682410" watchObservedRunningTime="2026-10-30 21:54:26.598341971 +0000 UTC m=+44.160754815"
Oct 30 21:54:27.458941 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:27.458902    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-8727r" event={"ID":"17da628d-31f5-456c-ad37-52c884d22cf4","Type":"ContainerStarted","Data":"b2efc992ac407bc9453bf0af937e0cb8c5c387b45266522aea0db609a4f5c9b8"}
Oct 30 21:54:27.459465 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:27.458967    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-8727r" event={"ID":"17da628d-31f5-456c-ad37-52c884d22cf4","Type":"ContainerStarted","Data":"0d169c8c4e7bd9c99eaa629483f80dbbc2f3622cc0dea9232d0c8ebf773b01d9"}
Oct 30 21:54:27.459465 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:27.459007    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-8727r" event={"ID":"17da628d-31f5-456c-ad37-52c884d22cf4","Type":"ContainerStarted","Data":"83508ce878512c91073366b4ebedb367a384a986518d2347d0f760c70d456d22"}
Oct 30 21:54:27.461303 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:27.461276    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-df5jt" event={"ID":"7ddfa67d-20a0-473d-937f-25fd8e0ba80e","Type":"ContainerStarted","Data":"39dc66cb4855808d684e3bbc15770cc4cf807a2c4293f2f33a93dc5fc77995b2"}
Oct 30 21:54:27.461430 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:27.461326    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-df5jt" event={"ID":"7ddfa67d-20a0-473d-937f-25fd8e0ba80e","Type":"ContainerStarted","Data":"1ef158b1220583b40c772af8b8885879d6df674b9ffa02b8899f1b72ef83898a"}
Oct 30 21:54:27.574117 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:27.573881    2577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-df5jt" podStartSLOduration=2.469898352 podStartE2EDuration="3.573865953s" totalImagesPullingTime="1.021582967s" totalInitContainerRuntime="82.384634ms" isStatefulPod=true podCreationTimestamp="2026-10-30 21:54:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:54:27.573611301 +0000 UTC m=+45.136024165" watchObservedRunningTime="2026-10-30 21:54:27.573865953 +0000 UTC m=+45.136278797"
Oct 30 21:54:27.574551 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:27.574525    2577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-8727r" podStartSLOduration=2.307147553 podStartE2EDuration="3.574513883s" totalImagesPullingTime="1.26736633s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:54:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:54:27.51726904 +0000 UTC m=+45.079681910" watchObservedRunningTime="2026-10-30 21:54:27.574513883 +0000 UTC m=+45.136926725"
Oct 30 21:54:27.860305 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:27.860274    2577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-868d6f78df-nx2k6"]
Oct 30 21:54:27.863150 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:27.863128    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-868d6f78df-nx2k6"
Oct 30 21:54:27.900187 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:27.900155    2577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-868d6f78df-nx2k6"]
Oct 30 21:54:27.914388 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:27.914352    2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\""
Oct 30 21:54:27.914539 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:27.914419    2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\""
Oct 30 21:54:27.914539 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:27.914434    2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\""
Oct 30 21:54:27.914539 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:27.914437    2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-772zc\""
Oct 30 21:54:27.914676 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:27.914565    2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\""
Oct 30 21:54:27.914676 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:27.914576    2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-8c44nd500k9ok\""
Oct 30 21:54:27.914676 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:27.914628    2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\""
Oct 30 21:54:27.990210 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:27.990172    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/fbd8f09b-9962-4910-8b25-267cbbb3ea5d-secret-grpc-tls\") pod \"thanos-querier-868d6f78df-nx2k6\" (UID: \"fbd8f09b-9962-4910-8b25-267cbbb3ea5d\") " pod="openshift-monitoring/thanos-querier-868d6f78df-nx2k6"
Oct 30 21:54:27.990377 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:27.990231    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/fbd8f09b-9962-4910-8b25-267cbbb3ea5d-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-868d6f78df-nx2k6\" (UID: \"fbd8f09b-9962-4910-8b25-267cbbb3ea5d\") " pod="openshift-monitoring/thanos-querier-868d6f78df-nx2k6"
Oct 30 21:54:27.990377 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:27.990285    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zzw6w\" (UniqueName: \"kubernetes.io/projected/fbd8f09b-9962-4910-8b25-267cbbb3ea5d-kube-api-access-zzw6w\") pod \"thanos-querier-868d6f78df-nx2k6\" (UID: \"fbd8f09b-9962-4910-8b25-267cbbb3ea5d\") " pod="openshift-monitoring/thanos-querier-868d6f78df-nx2k6"
Oct 30 21:54:27.990516 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:27.990409    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/fbd8f09b-9962-4910-8b25-267cbbb3ea5d-secret-thanos-querier-tls\") pod \"thanos-querier-868d6f78df-nx2k6\" (UID: \"fbd8f09b-9962-4910-8b25-267cbbb3ea5d\") " pod="openshift-monitoring/thanos-querier-868d6f78df-nx2k6"
Oct 30 21:54:27.990516 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:27.990470    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/fbd8f09b-9962-4910-8b25-267cbbb3ea5d-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-868d6f78df-nx2k6\" (UID: \"fbd8f09b-9962-4910-8b25-267cbbb3ea5d\") " pod="openshift-monitoring/thanos-querier-868d6f78df-nx2k6"
Oct 30 21:54:27.990612 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:27.990513    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fbd8f09b-9962-4910-8b25-267cbbb3ea5d-metrics-client-ca\") pod \"thanos-querier-868d6f78df-nx2k6\" (UID: \"fbd8f09b-9962-4910-8b25-267cbbb3ea5d\") " pod="openshift-monitoring/thanos-querier-868d6f78df-nx2k6"
Oct 30 21:54:27.990612 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:27.990562    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/fbd8f09b-9962-4910-8b25-267cbbb3ea5d-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-868d6f78df-nx2k6\" (UID: \"fbd8f09b-9962-4910-8b25-267cbbb3ea5d\") " pod="openshift-monitoring/thanos-querier-868d6f78df-nx2k6"
Oct 30 21:54:27.990612 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:27.990601    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fbd8f09b-9962-4910-8b25-267cbbb3ea5d-tmp\") pod \"thanos-querier-868d6f78df-nx2k6\" (UID: \"fbd8f09b-9962-4910-8b25-267cbbb3ea5d\") " pod="openshift-monitoring/thanos-querier-868d6f78df-nx2k6"
Oct 30 21:54:27.990725 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:27.990625    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/fbd8f09b-9962-4910-8b25-267cbbb3ea5d-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-868d6f78df-nx2k6\" (UID: \"fbd8f09b-9962-4910-8b25-267cbbb3ea5d\") " pod="openshift-monitoring/thanos-querier-868d6f78df-nx2k6"
Oct 30 21:54:28.091302 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:28.091267    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/fbd8f09b-9962-4910-8b25-267cbbb3ea5d-secret-thanos-querier-tls\") pod \"thanos-querier-868d6f78df-nx2k6\" (UID: \"fbd8f09b-9962-4910-8b25-267cbbb3ea5d\") " pod="openshift-monitoring/thanos-querier-868d6f78df-nx2k6"
Oct 30 21:54:28.091302 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:28.091308    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/fbd8f09b-9962-4910-8b25-267cbbb3ea5d-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-868d6f78df-nx2k6\" (UID: \"fbd8f09b-9962-4910-8b25-267cbbb3ea5d\") " pod="openshift-monitoring/thanos-querier-868d6f78df-nx2k6"
Oct 30 21:54:28.091533 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:28.091331    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fbd8f09b-9962-4910-8b25-267cbbb3ea5d-metrics-client-ca\") pod \"thanos-querier-868d6f78df-nx2k6\" (UID: \"fbd8f09b-9962-4910-8b25-267cbbb3ea5d\") " pod="openshift-monitoring/thanos-querier-868d6f78df-nx2k6"
Oct 30 21:54:28.091533 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:28.091401    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/fbd8f09b-9962-4910-8b25-267cbbb3ea5d-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-868d6f78df-nx2k6\" (UID: \"fbd8f09b-9962-4910-8b25-267cbbb3ea5d\") " pod="openshift-monitoring/thanos-querier-868d6f78df-nx2k6"
Oct 30 21:54:28.091533 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:28.091431    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fbd8f09b-9962-4910-8b25-267cbbb3ea5d-tmp\") pod \"thanos-querier-868d6f78df-nx2k6\" (UID: \"fbd8f09b-9962-4910-8b25-267cbbb3ea5d\") " pod="openshift-monitoring/thanos-querier-868d6f78df-nx2k6"
Oct 30 21:54:28.091533 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:28.091450    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/fbd8f09b-9962-4910-8b25-267cbbb3ea5d-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-868d6f78df-nx2k6\" (UID: \"fbd8f09b-9962-4910-8b25-267cbbb3ea5d\") " pod="openshift-monitoring/thanos-querier-868d6f78df-nx2k6"
Oct 30 21:54:28.091718 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:28.091611    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/fbd8f09b-9962-4910-8b25-267cbbb3ea5d-secret-grpc-tls\") pod \"thanos-querier-868d6f78df-nx2k6\" (UID: \"fbd8f09b-9962-4910-8b25-267cbbb3ea5d\") " pod="openshift-monitoring/thanos-querier-868d6f78df-nx2k6"
Oct 30 21:54:28.091718 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:28.091678    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/fbd8f09b-9962-4910-8b25-267cbbb3ea5d-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-868d6f78df-nx2k6\" (UID: \"fbd8f09b-9962-4910-8b25-267cbbb3ea5d\") " pod="openshift-monitoring/thanos-querier-868d6f78df-nx2k6"
Oct 30 21:54:28.091718 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:28.091707    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zzw6w\" (UniqueName: \"kubernetes.io/projected/fbd8f09b-9962-4910-8b25-267cbbb3ea5d-kube-api-access-zzw6w\") pod \"thanos-querier-868d6f78df-nx2k6\" (UID: \"fbd8f09b-9962-4910-8b25-267cbbb3ea5d\") " pod="openshift-monitoring/thanos-querier-868d6f78df-nx2k6"
Oct 30 21:54:28.092371 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:28.092318    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fbd8f09b-9962-4910-8b25-267cbbb3ea5d-tmp\") pod \"thanos-querier-868d6f78df-nx2k6\" (UID: \"fbd8f09b-9962-4910-8b25-267cbbb3ea5d\") " pod="openshift-monitoring/thanos-querier-868d6f78df-nx2k6"
Oct 30 21:54:28.092581 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:28.092553    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fbd8f09b-9962-4910-8b25-267cbbb3ea5d-metrics-client-ca\") pod \"thanos-querier-868d6f78df-nx2k6\" (UID: \"fbd8f09b-9962-4910-8b25-267cbbb3ea5d\") " pod="openshift-monitoring/thanos-querier-868d6f78df-nx2k6"
Oct 30 21:54:28.096807 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:28.096779    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/fbd8f09b-9962-4910-8b25-267cbbb3ea5d-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-868d6f78df-nx2k6\" (UID: \"fbd8f09b-9962-4910-8b25-267cbbb3ea5d\") " pod="openshift-monitoring/thanos-querier-868d6f78df-nx2k6"
Oct 30 21:54:28.096917 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:28.096810    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/fbd8f09b-9962-4910-8b25-267cbbb3ea5d-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-868d6f78df-nx2k6\" (UID: \"fbd8f09b-9962-4910-8b25-267cbbb3ea5d\") " pod="openshift-monitoring/thanos-querier-868d6f78df-nx2k6"
Oct 30 21:54:28.096917 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:28.096807    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/fbd8f09b-9962-4910-8b25-267cbbb3ea5d-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-868d6f78df-nx2k6\" (UID: \"fbd8f09b-9962-4910-8b25-267cbbb3ea5d\") " pod="openshift-monitoring/thanos-querier-868d6f78df-nx2k6"
Oct 30 21:54:28.096917 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:28.096901    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/fbd8f09b-9962-4910-8b25-267cbbb3ea5d-secret-grpc-tls\") pod \"thanos-querier-868d6f78df-nx2k6\" (UID: \"fbd8f09b-9962-4910-8b25-267cbbb3ea5d\") " pod="openshift-monitoring/thanos-querier-868d6f78df-nx2k6"
Oct 30 21:54:28.097151 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:28.097133    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/fbd8f09b-9962-4910-8b25-267cbbb3ea5d-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-868d6f78df-nx2k6\" (UID: \"fbd8f09b-9962-4910-8b25-267cbbb3ea5d\") " pod="openshift-monitoring/thanos-querier-868d6f78df-nx2k6"
Oct 30 21:54:28.097448 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:28.097433    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/fbd8f09b-9962-4910-8b25-267cbbb3ea5d-secret-thanos-querier-tls\") pod \"thanos-querier-868d6f78df-nx2k6\" (UID: \"fbd8f09b-9962-4910-8b25-267cbbb3ea5d\") " pod="openshift-monitoring/thanos-querier-868d6f78df-nx2k6"
Oct 30 21:54:28.115182 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:28.115117    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zzw6w\" (UniqueName: \"kubernetes.io/projected/fbd8f09b-9962-4910-8b25-267cbbb3ea5d-kube-api-access-zzw6w\") pod \"thanos-querier-868d6f78df-nx2k6\" (UID: \"fbd8f09b-9962-4910-8b25-267cbbb3ea5d\") " pod="openshift-monitoring/thanos-querier-868d6f78df-nx2k6"
Oct 30 21:54:28.179635 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:28.179600    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-868d6f78df-nx2k6"
Oct 30 21:54:28.504278 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:28.502880    2577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-868d6f78df-nx2k6"]
Oct 30 21:54:28.512743 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:54:28.512715    2577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfbd8f09b_9962_4910_8b25_267cbbb3ea5d.slice/crio-bfb89131d151d9c07894bf934194c2ebb93199fbe0f6297f748d4cdf54d68975 WatchSource:0}: Error finding container bfb89131d151d9c07894bf934194c2ebb93199fbe0f6297f748d4cdf54d68975: Status 404 returned error can't find the container with id bfb89131d151d9c07894bf934194c2ebb93199fbe0f6297f748d4cdf54d68975
Oct 30 21:54:29.313280 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:29.313244    2577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-77f4f76594-htsvq"]
Oct 30 21:54:29.315541 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:29.315513    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-77f4f76594-htsvq"
Oct 30 21:54:29.349092 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:29.349053    2577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-77f4f76594-htsvq"]
Oct 30 21:54:29.361332 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:29.361299    2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-l4wkb\""
Oct 30 21:54:29.361499 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:29.361446    2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\""
Oct 30 21:54:29.469179 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:29.469145    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-868d6f78df-nx2k6" event={"ID":"fbd8f09b-9962-4910-8b25-267cbbb3ea5d","Type":"ContainerStarted","Data":"bfb89131d151d9c07894bf934194c2ebb93199fbe0f6297f748d4cdf54d68975"}
Oct 30 21:54:29.500773 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:29.500729    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w78hr\" (UniqueName: \"kubernetes.io/projected/9d2cced5-9ceb-421f-b831-8370a5ee0ca9-kube-api-access-w78hr\") pod \"monitoring-plugin-77f4f76594-htsvq\" (UID: \"9d2cced5-9ceb-421f-b831-8370a5ee0ca9\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-htsvq"
Oct 30 21:54:29.500971 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:29.500932    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/9d2cced5-9ceb-421f-b831-8370a5ee0ca9-monitoring-plugin-cert\") pod \"monitoring-plugin-77f4f76594-htsvq\" (UID: \"9d2cced5-9ceb-421f-b831-8370a5ee0ca9\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-htsvq"
Oct 30 21:54:29.602254 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:29.602174    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/9d2cced5-9ceb-421f-b831-8370a5ee0ca9-monitoring-plugin-cert\") pod \"monitoring-plugin-77f4f76594-htsvq\" (UID: \"9d2cced5-9ceb-421f-b831-8370a5ee0ca9\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-htsvq"
Oct 30 21:54:29.602254 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:29.602227    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w78hr\" (UniqueName: \"kubernetes.io/projected/9d2cced5-9ceb-421f-b831-8370a5ee0ca9-kube-api-access-w78hr\") pod \"monitoring-plugin-77f4f76594-htsvq\" (UID: \"9d2cced5-9ceb-421f-b831-8370a5ee0ca9\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-htsvq"
Oct 30 21:54:29.605055 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:29.605028    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/9d2cced5-9ceb-421f-b831-8370a5ee0ca9-monitoring-plugin-cert\") pod \"monitoring-plugin-77f4f76594-htsvq\" (UID: \"9d2cced5-9ceb-421f-b831-8370a5ee0ca9\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-htsvq"
Oct 30 21:54:29.629377 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:29.629341    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w78hr\" (UniqueName: \"kubernetes.io/projected/9d2cced5-9ceb-421f-b831-8370a5ee0ca9-kube-api-access-w78hr\") pod \"monitoring-plugin-77f4f76594-htsvq\" (UID: \"9d2cced5-9ceb-421f-b831-8370a5ee0ca9\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-htsvq"
Oct 30 21:54:29.928006 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:29.927907    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-77f4f76594-htsvq"
Oct 30 21:54:30.246871 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:30.246845    2577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-77f4f76594-htsvq"]
Oct 30 21:54:30.248349 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:54:30.248317    2577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9d2cced5_9ceb_421f_b831_8370a5ee0ca9.slice/crio-d65ef881919ac2a78210156713c755f348c9da7cd09bd6af266bbdfa0cb76498 WatchSource:0}: Error finding container d65ef881919ac2a78210156713c755f348c9da7cd09bd6af266bbdfa0cb76498: Status 404 returned error can't find the container with id d65ef881919ac2a78210156713c755f348c9da7cd09bd6af266bbdfa0cb76498
Oct 30 21:54:30.474897 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:30.474867    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-868d6f78df-nx2k6" event={"ID":"fbd8f09b-9962-4910-8b25-267cbbb3ea5d","Type":"ContainerStarted","Data":"7de608a0251c808c645d9b1f0bed05d3520d890036dfde3961bf5824fb483585"}
Oct 30 21:54:30.476198 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:30.476175    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-77f4f76594-htsvq" event={"ID":"9d2cced5-9ceb-421f-b831-8370a5ee0ca9","Type":"ContainerStarted","Data":"d65ef881919ac2a78210156713c755f348c9da7cd09bd6af266bbdfa0cb76498"}
Oct 30 21:54:31.483117 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:31.482824    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-868d6f78df-nx2k6" event={"ID":"fbd8f09b-9962-4910-8b25-267cbbb3ea5d","Type":"ContainerStarted","Data":"e768d8b7015332c8091feb656fd238ff7a5b878d321799e1941f7adff0b63c99"}
Oct 30 21:54:31.483117 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:31.482919    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-868d6f78df-nx2k6" event={"ID":"fbd8f09b-9962-4910-8b25-267cbbb3ea5d","Type":"ContainerStarted","Data":"56cb095cbaf92a1d7308cf3b82ecdf13d2c9382ebefa9ab9364bce700123ebcc"}
Oct 30 21:54:31.483117 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:31.482971    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-868d6f78df-nx2k6" event={"ID":"fbd8f09b-9962-4910-8b25-267cbbb3ea5d","Type":"ContainerStarted","Data":"0e80dfc2c78d0812a646c30d57d04c1472c9a25d6e1a1a3f03a99babe3408213"}
Oct 30 21:54:31.483117 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:31.483038    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-868d6f78df-nx2k6" event={"ID":"fbd8f09b-9962-4910-8b25-267cbbb3ea5d","Type":"ContainerStarted","Data":"467a6265f5fb08600ef8595bf74ad3abd624ede08c3e515aee114211741742c2"}
Oct 30 21:54:31.671884 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:31.671848    2577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Oct 30 21:54:31.675662 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:31.675644    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:54:31.712079 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:31.712011    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Oct 30 21:54:31.712079 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:31.712027    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Oct 30 21:54:31.712079 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:31.712053    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Oct 30 21:54:31.712338 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:31.712174    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Oct 30 21:54:31.713334 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:31.713315    2577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Oct 30 21:54:31.739534 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:31.739300    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Oct 30 21:54:31.739534 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:31.739459    2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Oct 30 21:54:31.739534 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:31.739473    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Oct 30 21:54:31.739534 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:31.739498    2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Oct 30 21:54:31.739534 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:31.739508    2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Oct 30 21:54:31.739850 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:31.739566    2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Oct 30 21:54:31.739959 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:31.739920    2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Oct 30 21:54:31.739959 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:31.739921    2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Oct 30 21:54:31.739959 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:31.739955    2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Oct 30 21:54:31.740130 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:31.740016    2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-xwsb9\""
Oct 30 21:54:31.740162 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:31.740136    2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Oct 30 21:54:31.740205 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:31.740192    2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Oct 30 21:54:31.740255 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:31.740243    2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-9k9apv9hqg1mm\""
Oct 30 21:54:31.823948 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:31.823908    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/bd93518f-2f96-4483-9e92-513109dd37cb-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"bd93518f-2f96-4483-9e92-513109dd37cb\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:54:31.824152 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:31.823962    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/bd93518f-2f96-4483-9e92-513109dd37cb-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"bd93518f-2f96-4483-9e92-513109dd37cb\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:54:31.824152 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:31.824007    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/bd93518f-2f96-4483-9e92-513109dd37cb-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"bd93518f-2f96-4483-9e92-513109dd37cb\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:54:31.824152 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:31.824069    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bd93518f-2f96-4483-9e92-513109dd37cb-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"bd93518f-2f96-4483-9e92-513109dd37cb\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:54:31.824152 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:31.824097    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/bd93518f-2f96-4483-9e92-513109dd37cb-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"bd93518f-2f96-4483-9e92-513109dd37cb\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:54:31.824309 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:31.824161    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/bd93518f-2f96-4483-9e92-513109dd37cb-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"bd93518f-2f96-4483-9e92-513109dd37cb\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:54:31.824309 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:31.824203    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/bd93518f-2f96-4483-9e92-513109dd37cb-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"bd93518f-2f96-4483-9e92-513109dd37cb\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:54:31.824309 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:31.824255    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bd93518f-2f96-4483-9e92-513109dd37cb-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"bd93518f-2f96-4483-9e92-513109dd37cb\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:54:31.824397 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:31.824330    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/bd93518f-2f96-4483-9e92-513109dd37cb-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"bd93518f-2f96-4483-9e92-513109dd37cb\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:54:31.824429 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:31.824406    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/bd93518f-2f96-4483-9e92-513109dd37cb-config-out\") pod \"prometheus-k8s-0\" (UID: \"bd93518f-2f96-4483-9e92-513109dd37cb\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:54:31.824462 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:31.824448    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/bd93518f-2f96-4483-9e92-513109dd37cb-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"bd93518f-2f96-4483-9e92-513109dd37cb\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:54:31.824496 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:31.824479    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/bd93518f-2f96-4483-9e92-513109dd37cb-web-config\") pod \"prometheus-k8s-0\" (UID: \"bd93518f-2f96-4483-9e92-513109dd37cb\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:54:31.824603 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:31.824543    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/bd93518f-2f96-4483-9e92-513109dd37cb-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"bd93518f-2f96-4483-9e92-513109dd37cb\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:54:31.824668 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:31.824614    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/bd93518f-2f96-4483-9e92-513109dd37cb-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"bd93518f-2f96-4483-9e92-513109dd37cb\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:54:31.824707 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:31.824668    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bd93518f-2f96-4483-9e92-513109dd37cb-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"bd93518f-2f96-4483-9e92-513109dd37cb\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:54:31.824744 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:31.824703    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bd93518f-2f96-4483-9e92-513109dd37cb-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"bd93518f-2f96-4483-9e92-513109dd37cb\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:54:31.824794 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:31.824748    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/bd93518f-2f96-4483-9e92-513109dd37cb-config\") pod \"prometheus-k8s-0\" (UID: \"bd93518f-2f96-4483-9e92-513109dd37cb\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:54:31.824841 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:31.824818    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/bd93518f-2f96-4483-9e92-513109dd37cb-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"bd93518f-2f96-4483-9e92-513109dd37cb\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:54:31.824896 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:31.824862    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/bd93518f-2f96-4483-9e92-513109dd37cb-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"bd93518f-2f96-4483-9e92-513109dd37cb\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:54:31.824942 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:31.824922    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qrt69\" (UniqueName: \"kubernetes.io/projected/bd93518f-2f96-4483-9e92-513109dd37cb-kube-api-access-qrt69\") pod \"prometheus-k8s-0\" (UID: \"bd93518f-2f96-4483-9e92-513109dd37cb\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:54:31.926216 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:31.926176    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/bd93518f-2f96-4483-9e92-513109dd37cb-config\") pod \"prometheus-k8s-0\" (UID: \"bd93518f-2f96-4483-9e92-513109dd37cb\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:54:31.926216 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:31.926219    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/bd93518f-2f96-4483-9e92-513109dd37cb-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"bd93518f-2f96-4483-9e92-513109dd37cb\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:54:31.926465 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:31.926244    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/bd93518f-2f96-4483-9e92-513109dd37cb-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"bd93518f-2f96-4483-9e92-513109dd37cb\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:54:31.926465 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:31.926259    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qrt69\" (UniqueName: \"kubernetes.io/projected/bd93518f-2f96-4483-9e92-513109dd37cb-kube-api-access-qrt69\") pod \"prometheus-k8s-0\" (UID: \"bd93518f-2f96-4483-9e92-513109dd37cb\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:54:31.926465 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:31.926301    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/bd93518f-2f96-4483-9e92-513109dd37cb-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"bd93518f-2f96-4483-9e92-513109dd37cb\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:54:31.926465 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:31.926317    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/bd93518f-2f96-4483-9e92-513109dd37cb-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"bd93518f-2f96-4483-9e92-513109dd37cb\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:54:31.926465 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:31.926334    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/bd93518f-2f96-4483-9e92-513109dd37cb-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"bd93518f-2f96-4483-9e92-513109dd37cb\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:54:31.926465 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:31.926350    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bd93518f-2f96-4483-9e92-513109dd37cb-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"bd93518f-2f96-4483-9e92-513109dd37cb\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:54:31.926465 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:31.926381    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/bd93518f-2f96-4483-9e92-513109dd37cb-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"bd93518f-2f96-4483-9e92-513109dd37cb\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:54:31.926465 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:31.926408    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/bd93518f-2f96-4483-9e92-513109dd37cb-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"bd93518f-2f96-4483-9e92-513109dd37cb\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:54:31.926465 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:31.926433    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/bd93518f-2f96-4483-9e92-513109dd37cb-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"bd93518f-2f96-4483-9e92-513109dd37cb\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:54:31.927194 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:31.927157    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/bd93518f-2f96-4483-9e92-513109dd37cb-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"bd93518f-2f96-4483-9e92-513109dd37cb\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:54:31.927396 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:31.927372    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bd93518f-2f96-4483-9e92-513109dd37cb-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"bd93518f-2f96-4483-9e92-513109dd37cb\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:54:31.927455 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:31.927433    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bd93518f-2f96-4483-9e92-513109dd37cb-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"bd93518f-2f96-4483-9e92-513109dd37cb\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:54:31.927521 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:31.927486    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/bd93518f-2f96-4483-9e92-513109dd37cb-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"bd93518f-2f96-4483-9e92-513109dd37cb\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:54:31.927574 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:31.927539    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/bd93518f-2f96-4483-9e92-513109dd37cb-config-out\") pod \"prometheus-k8s-0\" (UID: \"bd93518f-2f96-4483-9e92-513109dd37cb\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:54:31.927574 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:31.927567    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/bd93518f-2f96-4483-9e92-513109dd37cb-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"bd93518f-2f96-4483-9e92-513109dd37cb\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:54:31.928139 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:31.928114    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/bd93518f-2f96-4483-9e92-513109dd37cb-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"bd93518f-2f96-4483-9e92-513109dd37cb\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:54:31.928236 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:31.928136    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/bd93518f-2f96-4483-9e92-513109dd37cb-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"bd93518f-2f96-4483-9e92-513109dd37cb\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:54:31.928236 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:31.928186    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/bd93518f-2f96-4483-9e92-513109dd37cb-web-config\") pod \"prometheus-k8s-0\" (UID: \"bd93518f-2f96-4483-9e92-513109dd37cb\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:54:31.928342 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:31.928246    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/bd93518f-2f96-4483-9e92-513109dd37cb-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"bd93518f-2f96-4483-9e92-513109dd37cb\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:54:31.928342 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:31.928276    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/bd93518f-2f96-4483-9e92-513109dd37cb-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"bd93518f-2f96-4483-9e92-513109dd37cb\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:54:31.928342 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:31.928307    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bd93518f-2f96-4483-9e92-513109dd37cb-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"bd93518f-2f96-4483-9e92-513109dd37cb\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:54:31.928342 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:31.928337    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bd93518f-2f96-4483-9e92-513109dd37cb-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"bd93518f-2f96-4483-9e92-513109dd37cb\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:54:31.929427 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:31.929105    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bd93518f-2f96-4483-9e92-513109dd37cb-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"bd93518f-2f96-4483-9e92-513109dd37cb\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:54:31.929790 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:31.929757    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/bd93518f-2f96-4483-9e92-513109dd37cb-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"bd93518f-2f96-4483-9e92-513109dd37cb\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:54:31.930072 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:31.930045    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bd93518f-2f96-4483-9e92-513109dd37cb-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"bd93518f-2f96-4483-9e92-513109dd37cb\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:54:31.930289 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:31.930264    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/bd93518f-2f96-4483-9e92-513109dd37cb-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"bd93518f-2f96-4483-9e92-513109dd37cb\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:54:31.930601 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:31.930581    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/bd93518f-2f96-4483-9e92-513109dd37cb-config\") pod \"prometheus-k8s-0\" (UID: \"bd93518f-2f96-4483-9e92-513109dd37cb\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:54:31.931006 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:31.930962    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bd93518f-2f96-4483-9e92-513109dd37cb-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"bd93518f-2f96-4483-9e92-513109dd37cb\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:54:31.931096 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:31.931074    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/bd93518f-2f96-4483-9e92-513109dd37cb-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"bd93518f-2f96-4483-9e92-513109dd37cb\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:54:31.931415 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:31.931351    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/bd93518f-2f96-4483-9e92-513109dd37cb-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"bd93518f-2f96-4483-9e92-513109dd37cb\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:54:31.931415 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:31.931371    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/bd93518f-2f96-4483-9e92-513109dd37cb-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"bd93518f-2f96-4483-9e92-513109dd37cb\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:54:31.931973 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:31.931938    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/bd93518f-2f96-4483-9e92-513109dd37cb-web-config\") pod \"prometheus-k8s-0\" (UID: \"bd93518f-2f96-4483-9e92-513109dd37cb\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:54:31.932102 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:31.932006    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/bd93518f-2f96-4483-9e92-513109dd37cb-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"bd93518f-2f96-4483-9e92-513109dd37cb\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:54:31.932102 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:31.932060    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/bd93518f-2f96-4483-9e92-513109dd37cb-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"bd93518f-2f96-4483-9e92-513109dd37cb\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:54:31.932593 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:31.932573    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/bd93518f-2f96-4483-9e92-513109dd37cb-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"bd93518f-2f96-4483-9e92-513109dd37cb\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:54:31.932681 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:31.932664    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/bd93518f-2f96-4483-9e92-513109dd37cb-config-out\") pod \"prometheus-k8s-0\" (UID: \"bd93518f-2f96-4483-9e92-513109dd37cb\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:54:31.933298 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:31.933282    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/bd93518f-2f96-4483-9e92-513109dd37cb-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"bd93518f-2f96-4483-9e92-513109dd37cb\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:54:31.952104 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:31.952076    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qrt69\" (UniqueName: \"kubernetes.io/projected/bd93518f-2f96-4483-9e92-513109dd37cb-kube-api-access-qrt69\") pod \"prometheus-k8s-0\" (UID: \"bd93518f-2f96-4483-9e92-513109dd37cb\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:54:31.999968 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:31.999937    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:54:32.331765 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:32.331735    2577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Oct 30 21:54:32.332451 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:54:32.332420    2577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbd93518f_2f96_4483_9e92_513109dd37cb.slice/crio-83f5c0d6735a2e35b4e005074f2c7f583f92ea7a94d680c9c430b8d1df568d4c WatchSource:0}: Error finding container 83f5c0d6735a2e35b4e005074f2c7f583f92ea7a94d680c9c430b8d1df568d4c: Status 404 returned error can't find the container with id 83f5c0d6735a2e35b4e005074f2c7f583f92ea7a94d680c9c430b8d1df568d4c
Oct 30 21:54:32.489389 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:32.489351    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-868d6f78df-nx2k6" event={"ID":"fbd8f09b-9962-4910-8b25-267cbbb3ea5d","Type":"ContainerStarted","Data":"8f42963730fdd09354c1aa1f13aa20b933ccaedfdd8379ccc43359ea8955ca0d"}
Oct 30 21:54:32.489850 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:32.489506    2577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-868d6f78df-nx2k6"
Oct 30 21:54:32.490932 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:32.490908    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-77f4f76594-htsvq" event={"ID":"9d2cced5-9ceb-421f-b831-8370a5ee0ca9","Type":"ContainerStarted","Data":"8f6b4bce281aee1dc11acf1372c3ca0a0ba6f85b62970ad43118d5b2305412c4"}
Oct 30 21:54:32.491110 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:32.491094    2577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-77f4f76594-htsvq"
Oct 30 21:54:32.492373 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:32.492350    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"bd93518f-2f96-4483-9e92-513109dd37cb","Type":"ContainerStarted","Data":"83f5c0d6735a2e35b4e005074f2c7f583f92ea7a94d680c9c430b8d1df568d4c"}
Oct 30 21:54:32.496315 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:32.496294    2577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-77f4f76594-htsvq"
Oct 30 21:54:32.574150 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:32.574098    2577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-868d6f78df-nx2k6" podStartSLOduration=2.960899813 podStartE2EDuration="5.574081433s" totalImagesPullingTime="2.61318162s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:54:27 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:54:32.572895286 +0000 UTC m=+50.135308131" watchObservedRunningTime="2026-10-30 21:54:32.574081433 +0000 UTC m=+50.136494276"
Oct 30 21:54:32.628905 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:32.628801    2577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-77f4f76594-htsvq" podStartSLOduration=2.303633098 podStartE2EDuration="3.628785502s" totalImagesPullingTime="1.325152404s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:54:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:54:32.627524083 +0000 UTC m=+50.189936946" watchObservedRunningTime="2026-10-30 21:54:32.628785502 +0000 UTC m=+50.191198345"
Oct 30 21:54:33.497319 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:33.497285    2577 generic.go:356] "Generic (PLEG): container finished" podID="bd93518f-2f96-4483-9e92-513109dd37cb" containerID="1afbd58205aa89f01dece6ea4aceecdb2c1a95d6f90583bfa5c329026ead89df" exitCode=0
Oct 30 21:54:33.497800 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:33.497368    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"bd93518f-2f96-4483-9e92-513109dd37cb","Type":"ContainerDied","Data":"1afbd58205aa89f01dece6ea4aceecdb2c1a95d6f90583bfa5c329026ead89df"}
Oct 30 21:54:35.456115 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:35.456079    2577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-fqmbc"
Oct 30 21:54:37.401343 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:37.401307    2577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-mn9vl"
Oct 30 21:54:37.518084 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:37.518043    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"bd93518f-2f96-4483-9e92-513109dd37cb","Type":"ContainerStarted","Data":"9915681fb0e03c6fd0b8260c96d61e8c11ce6f9d448cf02897ca8c9db3796828"}
Oct 30 21:54:37.518243 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:37.518139    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"bd93518f-2f96-4483-9e92-513109dd37cb","Type":"ContainerStarted","Data":"4b5654280a0b3c41ca38a375f20091c72976104b1839d8f9f2405f57b1570677"}
Oct 30 21:54:37.518243 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:37.518192    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"bd93518f-2f96-4483-9e92-513109dd37cb","Type":"ContainerStarted","Data":"d76784ca45eef8daaa0b9110107077e348d4d174313732f2cc153bbbff9d32eb"}
Oct 30 21:54:37.518243 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:37.518234    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"bd93518f-2f96-4483-9e92-513109dd37cb","Type":"ContainerStarted","Data":"ac3251b59fa7c86209ea54ce01461512a94cf3ecfb802abd04895d4e0b226da3"}
Oct 30 21:54:37.518361 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:37.518279    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"bd93518f-2f96-4483-9e92-513109dd37cb","Type":"ContainerStarted","Data":"7f5d14c4283a270b49a30f65cd969f9059aa7d43f0f4a78590276fa736dae993"}
Oct 30 21:54:37.518361 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:37.518319    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"bd93518f-2f96-4483-9e92-513109dd37cb","Type":"ContainerStarted","Data":"fc4e2bd2f898fb51f973cbd22a4c415acb3e71bb1ccc34e58f5925bda9c48eda"}
Oct 30 21:54:37.588331 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:37.588280    2577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.705919024 podStartE2EDuration="6.588264156s" totalImagesPullingTime="3.86494474s" totalInitContainerRuntime="17.400392ms" isStatefulPod=false podCreationTimestamp="2026-10-30 21:54:31 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:54:37.586567618 +0000 UTC m=+55.148980463" watchObservedRunningTime="2026-10-30 21:54:37.588264156 +0000 UTC m=+55.150676999"
Oct 30 21:54:38.504003 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:38.503961    2577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-868d6f78df-nx2k6"
Oct 30 21:54:42.001191 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:42.001138    2577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:54:47.866673 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:47.866626    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/532658b6-c86e-4feb-89cb-14a91316a7a0-metrics-certs\") pod \"network-metrics-daemon-n8xrh\" (UID: \"532658b6-c86e-4feb-89cb-14a91316a7a0\") " pod="openshift-multus/network-metrics-daemon-n8xrh"
Oct 30 21:54:47.867238 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:47.866730    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/af83b188-09cb-4621-a9da-be33856d19f0-original-pull-secret\") pod \"global-pull-secret-syncer-q7fwz\" (UID: \"af83b188-09cb-4621-a9da-be33856d19f0\") " pod="kube-system/global-pull-secret-syncer-q7fwz"
Oct 30 21:54:47.895115 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:47.895080    2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Oct 30 21:54:47.895286 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:47.895185    2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Oct 30 21:54:47.899516 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:47.899490    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/af83b188-09cb-4621-a9da-be33856d19f0-original-pull-secret\") pod \"global-pull-secret-syncer-q7fwz\" (UID: \"af83b188-09cb-4621-a9da-be33856d19f0\") " pod="kube-system/global-pull-secret-syncer-q7fwz"
Oct 30 21:54:47.899637 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:47.899487    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/532658b6-c86e-4feb-89cb-14a91316a7a0-metrics-certs\") pod \"network-metrics-daemon-n8xrh\" (UID: \"532658b6-c86e-4feb-89cb-14a91316a7a0\") " pod="openshift-multus/network-metrics-daemon-n8xrh"
Oct 30 21:54:47.968153 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:47.968116    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ljsqq\" (UniqueName: \"kubernetes.io/projected/32ae0d42-4b76-47b7-b9d6-26b09f00ba5d-kube-api-access-ljsqq\") pod \"network-check-target-fvbz8\" (UID: \"32ae0d42-4b76-47b7-b9d6-26b09f00ba5d\") " pod="openshift-network-diagnostics/network-check-target-fvbz8"
Oct 30 21:54:47.983225 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:47.983196    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Oct 30 21:54:48.003567 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:48.003542    2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Oct 30 21:54:48.011187 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:48.011163    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ljsqq\" (UniqueName: \"kubernetes.io/projected/32ae0d42-4b76-47b7-b9d6-26b09f00ba5d-kube-api-access-ljsqq\") pod \"network-check-target-fvbz8\" (UID: \"32ae0d42-4b76-47b7-b9d6-26b09f00ba5d\") " pod="openshift-network-diagnostics/network-check-target-fvbz8"
Oct 30 21:54:48.027708 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:48.027676    2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-2n5cw\""
Oct 30 21:54:48.033063 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:48.033037    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-n8xrh"
Oct 30 21:54:48.042155 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:48.042139    2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-check-target-dockercfg-vwffm\""
Oct 30 21:54:48.048376 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:48.048354    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fvbz8"
Oct 30 21:54:48.055754 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:48.055715    2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"global-pull-secret-syncer-dockercfg-cggx8\""
Oct 30 21:54:48.060614 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:48.060586    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-q7fwz"
Oct 30 21:54:48.331790 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:48.331766    2577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-n8xrh"]
Oct 30 21:54:48.332566 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:54:48.332530    2577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod532658b6_c86e_4feb_89cb_14a91316a7a0.slice/crio-9bd7faef61deec19be60215e6651eb7da94b352c4e82286d682c30c02b91f953 WatchSource:0}: Error finding container 9bd7faef61deec19be60215e6651eb7da94b352c4e82286d682c30c02b91f953: Status 404 returned error can't find the container with id 9bd7faef61deec19be60215e6651eb7da94b352c4e82286d682c30c02b91f953
Oct 30 21:54:48.352683 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:48.352659    2577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-fvbz8"]
Oct 30 21:54:48.353355 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:54:48.353332    2577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod32ae0d42_4b76_47b7_b9d6_26b09f00ba5d.slice/crio-4397165a20a420aac8ad6d0f822e75d8972b23e73db86d9ba549c829116c4480 WatchSource:0}: Error finding container 4397165a20a420aac8ad6d0f822e75d8972b23e73db86d9ba549c829116c4480: Status 404 returned error can't find the container with id 4397165a20a420aac8ad6d0f822e75d8972b23e73db86d9ba549c829116c4480
Oct 30 21:54:48.364670 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:48.364645    2577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-q7fwz"]
Oct 30 21:54:48.365103 ip-10-0-143-246 kubenswrapper[2577]: W1030 21:54:48.365083    2577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaf83b188_09cb_4621_a9da_be33856d19f0.slice/crio-0bc4cf08b77946fc12c13d79e800268cebb120d1de733fa971a67b52e17c9336 WatchSource:0}: Error finding container 0bc4cf08b77946fc12c13d79e800268cebb120d1de733fa971a67b52e17c9336: Status 404 returned error can't find the container with id 0bc4cf08b77946fc12c13d79e800268cebb120d1de733fa971a67b52e17c9336
Oct 30 21:54:48.559331 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:48.559294    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-fvbz8" event={"ID":"32ae0d42-4b76-47b7-b9d6-26b09f00ba5d","Type":"ContainerStarted","Data":"4397165a20a420aac8ad6d0f822e75d8972b23e73db86d9ba549c829116c4480"}
Oct 30 21:54:48.560384 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:48.560355    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-n8xrh" event={"ID":"532658b6-c86e-4feb-89cb-14a91316a7a0","Type":"ContainerStarted","Data":"9bd7faef61deec19be60215e6651eb7da94b352c4e82286d682c30c02b91f953"}
Oct 30 21:54:48.561457 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:48.561436    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-q7fwz" event={"ID":"af83b188-09cb-4621-a9da-be33856d19f0","Type":"ContainerStarted","Data":"0bc4cf08b77946fc12c13d79e800268cebb120d1de733fa971a67b52e17c9336"}
Oct 30 21:54:49.569003 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:49.568466    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-n8xrh" event={"ID":"532658b6-c86e-4feb-89cb-14a91316a7a0","Type":"ContainerStarted","Data":"1c4b2e71d8619c962e94f51edd33921856325a0a0cdcd3ce61c1a5c6acbf13e2"}
Oct 30 21:54:49.569003 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:49.568528    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-n8xrh" event={"ID":"532658b6-c86e-4feb-89cb-14a91316a7a0","Type":"ContainerStarted","Data":"7e750b99dd1783926afd1942bee33efd7074205aba985692373c75eb6624776f"}
Oct 30 21:54:49.614700 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:49.614563    2577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-n8xrh" podStartSLOduration=65.615318961 podStartE2EDuration="1m6.614548775s" totalImagesPullingTime="999.229814ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:53:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:54:49.614081143 +0000 UTC m=+67.176493989" watchObservedRunningTime="2026-10-30 21:54:49.614548775 +0000 UTC m=+67.176961617"
Oct 30 21:54:51.578135 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:51.578101    2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-fvbz8" event={"ID":"32ae0d42-4b76-47b7-b9d6-26b09f00ba5d","Type":"ContainerStarted","Data":"4c30711e688a054afc74c14497d7c09048fd63936196d0b3b28cade33306bd31"}
Oct 30 21:54:51.578614 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:51.578358    2577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-fvbz8"
Oct 30 21:54:51.621633 ip-10-0-143-246 kubenswrapper[2577]: I1030 21:54:51.621582    2577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-fvbz8" podStartSLOduration=65.480776851 podStartE2EDuration="1m8.621568903s" totalImagesPullingTime="3.140792052s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:53:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:54:51.62050611 +0000 UTC m=+69.182918952" watchObservedRunningTime="2026-10-30 21:54:51.621568903 +0000 UTC m=+69.183981747"
Oct 30 21:54:51.680407 ip-10-0-143-246 kubenswrapper[2577]: E1030 21:54:51.680368    2577 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6jppj" containerName="extractor"
Oct 30 21:54:53.177187 ip-10-0-143-246 systemd[1]: Stopping Kubernetes Kubelet...
Oct 30 21:54:53.195409 ip-10-0-143-246 systemd[1]: kubelet.service: Deactivated successfully.
Oct 30 21:54:53.195587 ip-10-0-143-246 systemd[1]: Stopped Kubernetes Kubelet.
Oct 30 21:54:53.195782 ip-10-0-143-246 systemd[1]: kubelet.service: Consumed 4.280s CPU time.
Oct 30 21:54:53.206594 ip-10-0-143-246 systemd[1]: Starting Kubernetes Kubelet...
Oct 30 21:54:53.272171 ip-10-0-143-246 kubenswrapper[6133]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 30 21:54:53.272171 ip-10-0-143-246 kubenswrapper[6133]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Oct 30 21:54:53.272171 ip-10-0-143-246 kubenswrapper[6133]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 30 21:54:53.272171 ip-10-0-143-246 kubenswrapper[6133]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 30 21:54:53.274384 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.274281    6133 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 21:54:53.274384 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.274383    6133 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 21:54:53.274384 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.274387    6133 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 21:54:53.274384 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.274390    6133 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 21:54:53.274535 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.274393    6133 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 21:54:53.274535 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.274397    6133 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 21:54:53.274535 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.274399    6133 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 21:54:53.274535 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.274403    6133 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 21:54:53.274535 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.274406    6133 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 21:54:53.274535 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.274409    6133 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 21:54:53.274535 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.274412    6133 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 21:54:53.274535 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.274414    6133 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 21:54:53.274535 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.274417    6133 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 21:54:53.274535 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.274420    6133 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 21:54:53.274535 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.274423    6133 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 21:54:53.274535 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.274426    6133 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 21:54:53.274535 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.274429    6133 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 21:54:53.274535 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.274431    6133 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 21:54:53.274535 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.274434    6133 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 21:54:53.274535 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.274437    6133 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 21:54:53.274535 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.274441    6133 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 21:54:53.274535 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.274446    6133 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 21:54:53.274535 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.274451    6133 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 21:54:53.274535 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.274467    6133 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 21:54:53.275047 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.274471    6133 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 21:54:53.275047 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.274474    6133 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 21:54:53.275047 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.274478    6133 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 21:54:53.275047 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.274483    6133 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 21:54:53.275047 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.274486    6133 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 21:54:53.275047 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.274489    6133 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 21:54:53.275047 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.274493    6133 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 21:54:53.275047 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.274495    6133 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 21:54:53.275047 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.274498    6133 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 21:54:53.275047 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.274501    6133 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 21:54:53.275047 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.274504    6133 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 21:54:53.275047 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.274507    6133 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 21:54:53.275047 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.274510    6133 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 21:54:53.275047 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.274512    6133 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 21:54:53.275047 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.274515    6133 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 21:54:53.275047 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.274519    6133 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 21:54:53.275047 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.274523    6133 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 21:54:53.275047 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.274527    6133 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 21:54:53.275047 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.274530    6133 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 21:54:53.275514 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.274533    6133 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 21:54:53.275514 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.274536    6133 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 21:54:53.275514 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.274539    6133 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 21:54:53.275514 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.274542    6133 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 21:54:53.275514 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.274545    6133 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 21:54:53.275514 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.274548    6133 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 21:54:53.275514 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.274550    6133 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 21:54:53.275514 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.274553    6133 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 21:54:53.275514 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.274556    6133 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 21:54:53.275514 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.274559    6133 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 21:54:53.275514 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.274561    6133 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 21:54:53.275514 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.274564    6133 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 21:54:53.275514 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.274567    6133 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 21:54:53.275514 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.274570    6133 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 21:54:53.275514 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.274574    6133 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 21:54:53.275514 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.274577    6133 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 21:54:53.275514 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.274580    6133 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 21:54:53.275514 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.274583    6133 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 21:54:53.275514 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.274586    6133 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 21:54:53.275982 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.274589    6133 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 21:54:53.275982 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.274591    6133 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 21:54:53.275982 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.274594    6133 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 21:54:53.275982 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.274597    6133 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 21:54:53.275982 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.274600    6133 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 21:54:53.275982 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.274603    6133 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 21:54:53.275982 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.274606    6133 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 21:54:53.275982 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.274608    6133 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 21:54:53.275982 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.274611    6133 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 21:54:53.275982 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.274613    6133 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 21:54:53.275982 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.274616    6133 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 21:54:53.275982 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.274618    6133 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 21:54:53.275982 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.274621    6133 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 21:54:53.275982 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.274623    6133 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 21:54:53.275982 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.274626    6133 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 21:54:53.275982 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.274628    6133 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 21:54:53.275982 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.274631    6133 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 21:54:53.275982 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.274633    6133 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 21:54:53.275982 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.274636    6133 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 21:54:53.275982 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.274638    6133 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 21:54:53.276549 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.274641    6133 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 21:54:53.276549 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.274643    6133 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 21:54:53.276549 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.274646    6133 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 21:54:53.276549 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.274649    6133 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 21:54:53.276549 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.274651    6133 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 21:54:53.276549 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.274654    6133 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 21:54:53.276549 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.274657    6133 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 21:54:53.276549 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.274662    6133 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 21:54:53.276549 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.274664    6133 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 21:54:53.276549 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.274667    6133 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 21:54:53.276549 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.274670    6133 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 21:54:53.276549 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.274673    6133 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 21:54:53.276549 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.274675    6133 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 21:54:53.276549 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.274678    6133 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 21:54:53.276549 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.274682    6133 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 21:54:53.276549 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.274684    6133 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 21:54:53.276549 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.274687    6133 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 21:54:53.276549 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.274689    6133 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 21:54:53.276549 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.274692    6133 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 21:54:53.276549 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.274695    6133 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 21:54:53.277072 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.274697    6133 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 21:54:53.277072 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.274700    6133 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 21:54:53.277072 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.274703    6133 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 21:54:53.277072 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.274705    6133 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 21:54:53.277072 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.275517    6133 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 21:54:53.277072 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.275525    6133 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 21:54:53.277072 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.275528    6133 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 21:54:53.277072 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.275532    6133 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 21:54:53.277072 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.275535    6133 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 21:54:53.277072 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.275538    6133 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 21:54:53.277072 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.275540    6133 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 21:54:53.277072 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.275543    6133 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 21:54:53.277072 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.275546    6133 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 21:54:53.277072 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.275549    6133 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 21:54:53.277072 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.275552    6133 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 21:54:53.277072 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.275555    6133 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 21:54:53.277072 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.275557    6133 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 21:54:53.277072 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.275560    6133 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 21:54:53.277072 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.275563    6133 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 21:54:53.277545 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.275569    6133 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 21:54:53.277545 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.275572    6133 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 21:54:53.277545 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.275575    6133 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 21:54:53.277545 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.275578    6133 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 21:54:53.277545 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.275580    6133 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 21:54:53.277545 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.275583    6133 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 21:54:53.277545 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.275586    6133 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 21:54:53.277545 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.275589    6133 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 21:54:53.277545 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.275592    6133 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 21:54:53.277545 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.275594    6133 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 21:54:53.277545 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.275597    6133 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 21:54:53.277545 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.275600    6133 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 21:54:53.277545 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.275602    6133 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 21:54:53.277545 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.275605    6133 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 21:54:53.277545 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.275608    6133 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 21:54:53.277545 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.275610    6133 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 21:54:53.277545 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.275613    6133 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 21:54:53.277545 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.275615    6133 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 21:54:53.277545 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.275618    6133 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 21:54:53.278032 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.275620    6133 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 21:54:53.278032 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.275623    6133 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 21:54:53.278032 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.275625    6133 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 21:54:53.278032 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.275628    6133 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 21:54:53.278032 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.275630    6133 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 21:54:53.278032 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.275633    6133 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 21:54:53.278032 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.275636    6133 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 21:54:53.278032 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.275639    6133 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 21:54:53.278032 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.275643    6133 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 21:54:53.278032 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.275647    6133 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 21:54:53.278032 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.275650    6133 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 21:54:53.278032 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.275654    6133 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 21:54:53.278032 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.275656    6133 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 21:54:53.278032 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.275660    6133 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 21:54:53.278032 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.275663    6133 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 21:54:53.278032 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.275673    6133 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 21:54:53.278032 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.275676    6133 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 21:54:53.278032 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.275679    6133 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 21:54:53.278032 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.275681    6133 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 21:54:53.278614 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.275684    6133 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 21:54:53.278614 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.275692    6133 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 21:54:53.278614 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.275695    6133 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 21:54:53.278614 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.275698    6133 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 21:54:53.278614 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.275701    6133 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 21:54:53.278614 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.275703    6133 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 21:54:53.278614 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.275706    6133 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 21:54:53.278614 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.275709    6133 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 21:54:53.278614 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.275712    6133 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 21:54:53.278614 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.275714    6133 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 21:54:53.278614 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.275717    6133 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 21:54:53.278614 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.275720    6133 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 21:54:53.278614 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.275722    6133 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 21:54:53.278614 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.275725    6133 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 21:54:53.278614 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.275728    6133 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 21:54:53.278614 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.275731    6133 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 21:54:53.278614 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.275735    6133 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 21:54:53.278614 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.275737    6133 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 21:54:53.278614 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.275740    6133 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 21:54:53.278614 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.275744    6133 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 21:54:53.279378 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.275747    6133 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 21:54:53.279378 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.275749    6133 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 21:54:53.279378 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.275751    6133 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 21:54:53.279378 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.275754    6133 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 21:54:53.279378 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.275757    6133 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 21:54:53.279378 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.275759    6133 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 21:54:53.279378 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.275763    6133 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 21:54:53.279378 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.275766    6133 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 21:54:53.279378 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.275769    6133 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 21:54:53.279378 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.275772    6133 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 21:54:53.279378 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.275775    6133 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 21:54:53.279378 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.275777    6133 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 21:54:53.279378 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.275780    6133 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 21:54:53.279378 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.275782    6133 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 21:54:53.279378 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.275785    6133 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 21:54:53.279378 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.275788    6133 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 21:54:53.279378 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.275790    6133 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 21:54:53.279378 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.275793    6133 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 21:54:53.279378 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.275795    6133 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 21:54:53.279378 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.275799    6133 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 21:54:53.279943 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.275802    6133 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 21:54:53.279943 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.275805    6133 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 21:54:53.279943 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.275807    6133 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 21:54:53.279943 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.275810    6133 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 21:54:53.279943 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.275813    6133 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 21:54:53.279943 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.275815    6133 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 21:54:53.279943 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.275818    6133 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 21:54:53.279943 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.275820    6133 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 21:54:53.279943 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.275823    6133 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 21:54:53.279943 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.275825    6133 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 21:54:53.279943 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.275827    6133 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 21:54:53.279943 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.275834    6133 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 21:54:53.279943 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.275837    6133 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 21:54:53.279943 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.276368    6133 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 21:54:53.279943 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.276374    6133 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 21:54:53.279943 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.276377    6133 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 21:54:53.279943 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.276380    6133 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 21:54:53.279943 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.276383    6133 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 21:54:53.279943 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.276388    6133 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 21:54:53.279943 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.276390    6133 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 21:54:53.280475 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.276393    6133 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 21:54:53.280475 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.276397    6133 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 21:54:53.280475 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.276400    6133 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 21:54:53.280475 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.276403    6133 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 21:54:53.280475 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.276405    6133 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 21:54:53.280475 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.276408    6133 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 21:54:53.280475 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.276411    6133 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 21:54:53.280475 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.276414    6133 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 21:54:53.280475 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.276416    6133 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 21:54:53.280475 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.276418    6133 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 21:54:53.280475 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.276421    6133 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 21:54:53.280475 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.276425    6133 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 21:54:53.280475 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.276428    6133 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 21:54:53.280475 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.276431    6133 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 21:54:53.280475 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.276434    6133 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 21:54:53.280475 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.276436    6133 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 21:54:53.280475 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.276439    6133 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 21:54:53.280475 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.276442    6133 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 21:54:53.280475 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.276444    6133 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 21:54:53.280475 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.276447    6133 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 21:54:53.280973 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.276449    6133 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 21:54:53.280973 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.276452    6133 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 21:54:53.280973 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.276454    6133 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 21:54:53.280973 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.276457    6133 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 21:54:53.280973 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.276460    6133 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 21:54:53.280973 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.276462    6133 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 21:54:53.280973 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.276466    6133 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 21:54:53.280973 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.276468    6133 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 21:54:53.280973 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.276471    6133 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 21:54:53.280973 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.276474    6133 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 21:54:53.280973 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.276478    6133 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 21:54:53.280973 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.276481    6133 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 21:54:53.280973 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.276483    6133 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 21:54:53.280973 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.276486    6133 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 21:54:53.280973 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.276489    6133 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 21:54:53.280973 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.276492    6133 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 21:54:53.280973 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.276495    6133 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 21:54:53.280973 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.276497    6133 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 21:54:53.280973 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.276500    6133 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 21:54:53.281483 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.276502    6133 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 21:54:53.281483 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.276505    6133 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 21:54:53.281483 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.276507    6133 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 21:54:53.281483 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.276510    6133 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 21:54:53.281483 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.276514    6133 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 21:54:53.281483 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.276516    6133 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 21:54:53.281483 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.276519    6133 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 21:54:53.281483 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.276522    6133 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 21:54:53.281483 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.276524    6133 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 21:54:53.281483 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.276527    6133 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 21:54:53.281483 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.276529    6133 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 21:54:53.281483 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.276532    6133 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 21:54:53.281483 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.276534    6133 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 21:54:53.281483 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.276537    6133 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 21:54:53.281483 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.276539    6133 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 21:54:53.281483 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.276542    6133 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 21:54:53.281483 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.276544    6133 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 21:54:53.281483 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.276547    6133 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 21:54:53.281483 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.276550    6133 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 21:54:53.281483 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.276553    6133 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 21:54:53.281980 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.276555    6133 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 21:54:53.281980 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.276564    6133 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 21:54:53.281980 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.276569    6133 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 21:54:53.281980 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.276573    6133 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 21:54:53.281980 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.276576    6133 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 21:54:53.281980 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.276578    6133 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 21:54:53.281980 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.276581    6133 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 21:54:53.281980 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.276584    6133 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 21:54:53.281980 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.276588    6133 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 21:54:53.281980 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.276592    6133 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 21:54:53.281980 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.276596    6133 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 21:54:53.281980 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.276600    6133 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 21:54:53.281980 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.276603    6133 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 21:54:53.281980 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.276606    6133 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 21:54:53.281980 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.276609    6133 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 21:54:53.281980 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.276612    6133 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 21:54:53.281980 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.276616    6133 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 21:54:53.281980 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.276618    6133 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 21:54:53.281980 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.276621    6133 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 21:54:53.282464 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.276623    6133 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 21:54:53.282464 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.276626    6133 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 21:54:53.282464 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.276628    6133 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 21:54:53.282464 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.276631    6133 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 21:54:53.282464 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.276633    6133 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 21:54:53.282464 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.276636    6133 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 21:54:53.282464 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.276639    6133 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 21:54:53.282464 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.276642    6133 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 21:54:53.282464 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.276644    6133 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 21:54:53.282464 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.276646    6133 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 21:54:53.282464 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.276649    6133 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 21:54:53.282464 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.276652    6133 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 21:54:53.282464 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.276654    6133 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 21:54:53.282464 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.276656    6133 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 21:54:53.282464 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.276659    6133 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 21:54:53.282464 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.276661    6133 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 21:54:53.282464 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.276666    6133 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 21:54:53.282464 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.276669    6133 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 21:54:53.282464 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.276671    6133 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 21:54:53.282464 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.276674    6133 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 21:54:53.282999 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.276678    6133 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 21:54:53.282999 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.276776    6133 flags.go:64] FLAG: --address="0.0.0.0"
Oct 30 21:54:53.282999 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.276791    6133 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Oct 30 21:54:53.282999 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.276798    6133 flags.go:64] FLAG: --anonymous-auth="true"
Oct 30 21:54:53.282999 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.276803    6133 flags.go:64] FLAG: --application-metrics-count-limit="100"
Oct 30 21:54:53.282999 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.276808    6133 flags.go:64] FLAG: --authentication-token-webhook="false"
Oct 30 21:54:53.282999 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.276811    6133 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Oct 30 21:54:53.282999 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.276816    6133 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Oct 30 21:54:53.282999 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.276820    6133 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Oct 30 21:54:53.282999 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.276825    6133 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Oct 30 21:54:53.282999 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.276829    6133 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Oct 30 21:54:53.282999 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.276833    6133 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Oct 30 21:54:53.282999 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.276836    6133 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Oct 30 21:54:53.282999 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.276840    6133 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Oct 30 21:54:53.282999 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.276843    6133 flags.go:64] FLAG: --cgroup-root=""
Oct 30 21:54:53.282999 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.276846    6133 flags.go:64] FLAG: --cgroups-per-qos="true"
Oct 30 21:54:53.282999 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.276849    6133 flags.go:64] FLAG: --client-ca-file=""
Oct 30 21:54:53.282999 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.276852    6133 flags.go:64] FLAG: --cloud-provider="external"
Oct 30 21:54:53.282999 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.276855    6133 flags.go:64] FLAG: --cluster-dns="[]"
Oct 30 21:54:53.282999 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.276860    6133 flags.go:64] FLAG: --cluster-domain=""
Oct 30 21:54:53.282999 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.276863    6133 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Oct 30 21:54:53.282999 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.276866    6133 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Oct 30 21:54:53.282999 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.276869    6133 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Oct 30 21:54:53.283561 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.276873    6133 flags.go:64] FLAG: --container-log-max-files="5"
Oct 30 21:54:53.283561 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.276877    6133 flags.go:64] FLAG: --container-log-max-size="10Mi"
Oct 30 21:54:53.283561 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.276880    6133 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Oct 30 21:54:53.283561 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.276884    6133 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Oct 30 21:54:53.283561 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.276887    6133 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Oct 30 21:54:53.283561 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.276891    6133 flags.go:64] FLAG: --contention-profiling="false"
Oct 30 21:54:53.283561 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.276894    6133 flags.go:64] FLAG: --cpu-cfs-quota="true"
Oct 30 21:54:53.283561 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.276897    6133 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Oct 30 21:54:53.283561 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.276901    6133 flags.go:64] FLAG: --cpu-manager-policy="none"
Oct 30 21:54:53.283561 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.276904    6133 flags.go:64] FLAG: --cpu-manager-policy-options=""
Oct 30 21:54:53.283561 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.276908    6133 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Oct 30 21:54:53.283561 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.276911    6133 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Oct 30 21:54:53.283561 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.276914    6133 flags.go:64] FLAG: --enable-debugging-handlers="true"
Oct 30 21:54:53.283561 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.276918    6133 flags.go:64] FLAG: --enable-load-reader="false"
Oct 30 21:54:53.283561 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.276921    6133 flags.go:64] FLAG: --enable-server="true"
Oct 30 21:54:53.283561 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.276924    6133 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Oct 30 21:54:53.283561 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.276928    6133 flags.go:64] FLAG: --event-burst="100"
Oct 30 21:54:53.283561 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.276931    6133 flags.go:64] FLAG: --event-qps="50"
Oct 30 21:54:53.283561 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.276936    6133 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Oct 30 21:54:53.283561 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.276939    6133 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Oct 30 21:54:53.283561 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.276942    6133 flags.go:64] FLAG: --eviction-hard=""
Oct 30 21:54:53.283561 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.276946    6133 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Oct 30 21:54:53.283561 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.276950    6133 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Oct 30 21:54:53.283561 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.276953    6133 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Oct 30 21:54:53.283561 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.276956    6133 flags.go:64] FLAG: --eviction-soft=""
Oct 30 21:54:53.284231 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.276959    6133 flags.go:64] FLAG: --eviction-soft-grace-period=""
Oct 30 21:54:53.284231 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.276962    6133 flags.go:64] FLAG: --exit-on-lock-contention="false"
Oct 30 21:54:53.284231 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.276965    6133 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Oct 30 21:54:53.284231 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.276968    6133 flags.go:64] FLAG: --experimental-mounter-path=""
Oct 30 21:54:53.284231 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.276971    6133 flags.go:64] FLAG: --fail-cgroupv1="true"
Oct 30 21:54:53.284231 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.276975    6133 flags.go:64] FLAG: --fail-swap-on="true"
Oct 30 21:54:53.284231 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.276978    6133 flags.go:64] FLAG: --feature-gates=""
Oct 30 21:54:53.284231 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.276982    6133 flags.go:64] FLAG: --file-check-frequency="20s"
Oct 30 21:54:53.284231 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.276999    6133 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Oct 30 21:54:53.284231 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.277004    6133 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Oct 30 21:54:53.284231 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.277009    6133 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Oct 30 21:54:53.284231 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.277014    6133 flags.go:64] FLAG: --healthz-port="10248"
Oct 30 21:54:53.284231 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.277019    6133 flags.go:64] FLAG: --help="false"
Oct 30 21:54:53.284231 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.277022    6133 flags.go:64] FLAG: --hostname-override="ip-10-0-143-246.ec2.internal"
Oct 30 21:54:53.284231 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.277026    6133 flags.go:64] FLAG: --housekeeping-interval="10s"
Oct 30 21:54:53.284231 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.277029    6133 flags.go:64] FLAG: --http-check-frequency="20s"
Oct 30 21:54:53.284231 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.277032    6133 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Oct 30 21:54:53.284231 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.277035    6133 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Oct 30 21:54:53.284231 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.277039    6133 flags.go:64] FLAG: --image-gc-high-threshold="85"
Oct 30 21:54:53.284231 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.277041    6133 flags.go:64] FLAG: --image-gc-low-threshold="80"
Oct 30 21:54:53.284231 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.277045    6133 flags.go:64] FLAG: --image-service-endpoint=""
Oct 30 21:54:53.284231 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.277048    6133 flags.go:64] FLAG: --kernel-memcg-notification="false"
Oct 30 21:54:53.284231 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.277051    6133 flags.go:64] FLAG: --kube-api-burst="100"
Oct 30 21:54:53.284816 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.277054    6133 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Oct 30 21:54:53.284816 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.277058    6133 flags.go:64] FLAG: --kube-api-qps="50"
Oct 30 21:54:53.284816 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.277062    6133 flags.go:64] FLAG: --kube-reserved=""
Oct 30 21:54:53.284816 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.277065    6133 flags.go:64] FLAG: --kube-reserved-cgroup=""
Oct 30 21:54:53.284816 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.277068    6133 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Oct 30 21:54:53.284816 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.277071    6133 flags.go:64] FLAG: --kubelet-cgroups=""
Oct 30 21:54:53.284816 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.277074    6133 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Oct 30 21:54:53.284816 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.277077    6133 flags.go:64] FLAG: --lock-file=""
Oct 30 21:54:53.284816 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.277081    6133 flags.go:64] FLAG: --log-cadvisor-usage="false"
Oct 30 21:54:53.284816 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.277084    6133 flags.go:64] FLAG: --log-flush-frequency="5s"
Oct 30 21:54:53.284816 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.277087    6133 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Oct 30 21:54:53.284816 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.277093    6133 flags.go:64] FLAG: --log-json-split-stream="false"
Oct 30 21:54:53.284816 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.277097    6133 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Oct 30 21:54:53.284816 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.277100    6133 flags.go:64] FLAG: --log-text-split-stream="false"
Oct 30 21:54:53.284816 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.277103    6133 flags.go:64] FLAG: --logging-format="text"
Oct 30 21:54:53.284816 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.277106    6133 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Oct 30 21:54:53.284816 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.277110    6133 flags.go:64] FLAG: --make-iptables-util-chains="true"
Oct 30 21:54:53.284816 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.277112    6133 flags.go:64] FLAG: --manifest-url=""
Oct 30 21:54:53.284816 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.277115    6133 flags.go:64] FLAG: --manifest-url-header=""
Oct 30 21:54:53.284816 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.277120    6133 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Oct 30 21:54:53.284816 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.277123    6133 flags.go:64] FLAG: --max-open-files="1000000"
Oct 30 21:54:53.284816 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.277128    6133 flags.go:64] FLAG: --max-pods="110"
Oct 30 21:54:53.284816 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.277131    6133 flags.go:64] FLAG: --maximum-dead-containers="-1"
Oct 30 21:54:53.284816 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.277134    6133 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Oct 30 21:54:53.284816 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.277137    6133 flags.go:64] FLAG: --memory-manager-policy="None"
Oct 30 21:54:53.285438 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.277140    6133 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Oct 30 21:54:53.285438 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.277143    6133 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Oct 30 21:54:53.285438 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.277145    6133 flags.go:64] FLAG: --node-ip="0.0.0.0"
Oct 30 21:54:53.285438 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.277148    6133 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Oct 30 21:54:53.285438 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.277156    6133 flags.go:64] FLAG: --node-status-max-images="50"
Oct 30 21:54:53.285438 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.277159    6133 flags.go:64] FLAG: --node-status-update-frequency="10s"
Oct 30 21:54:53.285438 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.277163    6133 flags.go:64] FLAG: --oom-score-adj="-999"
Oct 30 21:54:53.285438 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.277166    6133 flags.go:64] FLAG: --pod-cidr=""
Oct 30 21:54:53.285438 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.277168    6133 flags.go:64] FLAG: --pod-manifest-path=""
Oct 30 21:54:53.285438 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.277172    6133 flags.go:64] FLAG: --pod-max-pids="-1"
Oct 30 21:54:53.285438 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.277175    6133 flags.go:64] FLAG: --pods-per-core="0"
Oct 30 21:54:53.285438 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.277178    6133 flags.go:64] FLAG: --port="10250"
Oct 30 21:54:53.285438 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.277181    6133 flags.go:64] FLAG: --protect-kernel-defaults="false"
Oct 30 21:54:53.285438 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.277184    6133 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0134d5967aa4bbd7b"
Oct 30 21:54:53.285438 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.277187    6133 flags.go:64] FLAG: --qos-reserved=""
Oct 30 21:54:53.285438 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.277190    6133 flags.go:64] FLAG: --read-only-port="10255"
Oct 30 21:54:53.285438 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.277193    6133 flags.go:64] FLAG: --register-node="true"
Oct 30 21:54:53.285438 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.277196    6133 flags.go:64] FLAG: --register-with-taints=""
Oct 30 21:54:53.285438 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.277200    6133 flags.go:64] FLAG: --registry-burst="10"
Oct 30 21:54:53.285438 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.277203    6133 flags.go:64] FLAG: --registry-qps="5"
Oct 30 21:54:53.285438 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.277206    6133 flags.go:64] FLAG: --reserved-cpus=""
Oct 30 21:54:53.285438 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.277210    6133 flags.go:64] FLAG: --reserved-memory=""
Oct 30 21:54:53.285438 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.277214    6133 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Oct 30 21:54:53.285438 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.277217    6133 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Oct 30 21:54:53.285438 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.277220    6133 flags.go:64] FLAG: --rotate-certificates="false"
Oct 30 21:54:53.285438 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.277223    6133 flags.go:64] FLAG: --rotate-server-certificates="false"
Oct 30 21:54:53.286078 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.277226    6133 flags.go:64] FLAG: --runonce="false"
Oct 30 21:54:53.286078 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.277229    6133 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Oct 30 21:54:53.286078 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.277232    6133 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Oct 30 21:54:53.286078 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.277235    6133 flags.go:64] FLAG: --seccomp-default="false"
Oct 30 21:54:53.286078 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.277238    6133 flags.go:64] FLAG: --serialize-image-pulls="true"
Oct 30 21:54:53.286078 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.277241    6133 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Oct 30 21:54:53.286078 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.277244    6133 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Oct 30 21:54:53.286078 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.277248    6133 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Oct 30 21:54:53.286078 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.277251    6133 flags.go:64] FLAG: --storage-driver-password="root"
Oct 30 21:54:53.286078 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.277254    6133 flags.go:64] FLAG: --storage-driver-secure="false"
Oct 30 21:54:53.286078 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.277257    6133 flags.go:64] FLAG: --storage-driver-table="stats"
Oct 30 21:54:53.286078 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.277260    6133 flags.go:64] FLAG: --storage-driver-user="root"
Oct 30 21:54:53.286078 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.277264    6133 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Oct 30 21:54:53.286078 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.277267    6133 flags.go:64] FLAG: --sync-frequency="1m0s"
Oct 30 21:54:53.286078 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.277270    6133 flags.go:64] FLAG: --system-cgroups=""
Oct 30 21:54:53.286078 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.277273    6133 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Oct 30 21:54:53.286078 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.277279    6133 flags.go:64] FLAG: --system-reserved-cgroup=""
Oct 30 21:54:53.286078 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.277282    6133 flags.go:64] FLAG: --tls-cert-file=""
Oct 30 21:54:53.286078 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.277285    6133 flags.go:64] FLAG: --tls-cipher-suites="[]"
Oct 30 21:54:53.286078 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.277289    6133 flags.go:64] FLAG: --tls-min-version=""
Oct 30 21:54:53.286078 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.277292    6133 flags.go:64] FLAG: --tls-private-key-file=""
Oct 30 21:54:53.286078 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.277295    6133 flags.go:64] FLAG: --topology-manager-policy="none"
Oct 30 21:54:53.286078 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.277298    6133 flags.go:64] FLAG: --topology-manager-policy-options=""
Oct 30 21:54:53.286078 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.277301    6133 flags.go:64] FLAG: --topology-manager-scope="container"
Oct 30 21:54:53.286078 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.277304    6133 flags.go:64] FLAG: --v="2"
Oct 30 21:54:53.286681 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.277308    6133 flags.go:64] FLAG: --version="false"
Oct 30 21:54:53.286681 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.277313    6133 flags.go:64] FLAG: --vmodule=""
Oct 30 21:54:53.286681 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.277317    6133 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Oct 30 21:54:53.286681 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.277320    6133 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Oct 30 21:54:53.286681 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.277457    6133 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 21:54:53.286681 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.277461    6133 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 21:54:53.286681 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.277464    6133 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 21:54:53.286681 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.277467    6133 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 21:54:53.286681 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.277469    6133 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 21:54:53.286681 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.277472    6133 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 21:54:53.286681 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.277475    6133 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 21:54:53.286681 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.277477    6133 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 21:54:53.286681 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.277480    6133 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 21:54:53.286681 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.277482    6133 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 21:54:53.286681 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.277485    6133 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 21:54:53.286681 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.277487    6133 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 21:54:53.286681 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.277490    6133 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 21:54:53.286681 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.277493    6133 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 21:54:53.286681 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.277495    6133 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 21:54:53.286681 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.277497    6133 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 21:54:53.286681 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.277501    6133 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 21:54:53.287245 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.277503    6133 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 21:54:53.287245 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.277506    6133 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 21:54:53.287245 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.277508    6133 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 21:54:53.287245 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.277511    6133 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 21:54:53.287245 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.277513    6133 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 21:54:53.287245 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.277516    6133 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 21:54:53.287245 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.277519    6133 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 21:54:53.287245 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.277521    6133 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 21:54:53.287245 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.277524    6133 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 21:54:53.287245 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.277526    6133 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 21:54:53.287245 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.277529    6133 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 21:54:53.287245 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.277531    6133 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 21:54:53.287245 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.277534    6133 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 21:54:53.287245 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.277536    6133 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 21:54:53.287245 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.277539    6133 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 21:54:53.287245 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.277542    6133 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 21:54:53.287245 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.277545    6133 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 21:54:53.287245 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.277547    6133 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 21:54:53.287245 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.277550    6133 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 21:54:53.287245 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.277553    6133 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 21:54:53.287962 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.277556    6133 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 21:54:53.287962 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.277559    6133 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 21:54:53.287962 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.277562    6133 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 21:54:53.287962 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.277564    6133 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 21:54:53.287962 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.277567    6133 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 21:54:53.287962 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.277569    6133 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 21:54:53.287962 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.277572    6133 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 21:54:53.287962 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.277574    6133 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 21:54:53.287962 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.277578    6133 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 21:54:53.287962 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.277581    6133 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 21:54:53.287962 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.277584    6133 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 21:54:53.287962 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.277586    6133 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 21:54:53.287962 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.277589    6133 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 21:54:53.287962 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.277592    6133 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 21:54:53.287962 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.277594    6133 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 21:54:53.287962 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.277597    6133 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 21:54:53.287962 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.277599    6133 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 21:54:53.287962 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.277602    6133 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 21:54:53.287962 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.277606    6133 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 21:54:53.288589 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.277608    6133 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 21:54:53.288589 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.277612    6133 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 21:54:53.288589 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.277616    6133 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 21:54:53.288589 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.277619    6133 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 21:54:53.288589 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.277622    6133 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 21:54:53.288589 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.277624    6133 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 21:54:53.288589 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.277627    6133 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 21:54:53.288589 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.277630    6133 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 21:54:53.288589 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.277632    6133 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 21:54:53.288589 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.277635    6133 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 21:54:53.288589 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.277638    6133 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 21:54:53.288589 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.277641    6133 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 21:54:53.288589 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.277643    6133 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 21:54:53.288589 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.277646    6133 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 21:54:53.288589 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.277648    6133 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 21:54:53.288589 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.277651    6133 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 21:54:53.288589 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.277653    6133 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 21:54:53.288589 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.277656    6133 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 21:54:53.288589 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.277660    6133 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 21:54:53.289153 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.277663    6133 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 21:54:53.289153 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.277666    6133 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 21:54:53.289153 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.277670    6133 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 21:54:53.289153 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.277673    6133 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 21:54:53.289153 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.277675    6133 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 21:54:53.289153 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.277678    6133 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 21:54:53.289153 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.277681    6133 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 21:54:53.289153 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.277684    6133 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 21:54:53.289153 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.277687    6133 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 21:54:53.289153 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.277689    6133 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 21:54:53.289153 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.277691    6133 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 21:54:53.289153 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.277694    6133 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 21:54:53.289153 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.277698    6133 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 21:54:53.289153 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.277701    6133 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 21:54:53.289153 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.277704    6133 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 21:54:53.289153 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.277706    6133 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 21:54:53.289153 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.277709    6133 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 21:54:53.289153 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.277711    6133 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 21:54:53.289153 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.277714    6133 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 21:54:53.289153 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.277717    6133 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 21:54:53.289772 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.277719    6133 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 21:54:53.289772 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.277722    6133 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 21:54:53.289772 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.277725    6133 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 21:54:53.289772 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.277727    6133 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 21:54:53.289772 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.277730    6133 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 21:54:53.289772 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.277732    6133 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 21:54:53.289772 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.277735    6133 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 21:54:53.289772 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.277738    6133 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 21:54:53.289772 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.277740    6133 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 21:54:53.289772 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.277743    6133 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 21:54:53.289772 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.277746    6133 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 21:54:53.289772 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.277771    6133 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 30 21:54:53.289772 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.277806    6133 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 21:54:53.289772 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.277810    6133 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 21:54:53.289772 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.277815    6133 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 21:54:53.289772 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.277817    6133 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 21:54:53.290256 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.277820    6133 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 21:54:53.290256 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.277823    6133 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 21:54:53.290256 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.277825    6133 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 21:54:53.290256 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.277828    6133 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 21:54:53.290256 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.277831    6133 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 21:54:53.290256 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.277835    6133 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 21:54:53.290256 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.277838    6133 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 21:54:53.290256 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.277841    6133 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 21:54:53.290256 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.277843    6133 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 21:54:53.290256 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.277847    6133 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 21:54:53.290256 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.277850    6133 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 21:54:53.290256 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.277853    6133 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 21:54:53.290256 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.277855    6133 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 21:54:53.290256 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.277858    6133 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 21:54:53.290256 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.277861    6133 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 21:54:53.290256 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.277863    6133 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 21:54:53.290256 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.277866    6133 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 21:54:53.290256 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.277868    6133 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 21:54:53.290256 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.277871    6133 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 21:54:53.290769 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.277874    6133 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 21:54:53.290769 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.277876    6133 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 21:54:53.290769 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.277879    6133 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 21:54:53.290769 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.277882    6133 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 21:54:53.290769 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.277884    6133 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 21:54:53.290769 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.277887    6133 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 21:54:53.290769 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.277889    6133 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 21:54:53.290769 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.277892    6133 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 21:54:53.290769 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.277894    6133 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 21:54:53.290769 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.277897    6133 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 21:54:53.290769 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.277901    6133 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 21:54:53.290769 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.277907    6133 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 21:54:53.290769 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.277910    6133 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 21:54:53.290769 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.277914    6133 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 21:54:53.290769 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.277916    6133 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 21:54:53.290769 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.277919    6133 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 21:54:53.290769 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.277922    6133 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 21:54:53.290769 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.277925    6133 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 21:54:53.290769 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.277928    6133 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 21:54:53.291347 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.277930    6133 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 21:54:53.291347 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.277933    6133 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 21:54:53.291347 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.277936    6133 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 21:54:53.291347 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.277940    6133 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 21:54:53.291347 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.277943    6133 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 21:54:53.291347 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.277945    6133 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 21:54:53.291347 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.277948    6133 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 21:54:53.291347 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.277951    6133 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 21:54:53.291347 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.277954    6133 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 21:54:53.291347 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.277958    6133 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 21:54:53.291347 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.277961    6133 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 21:54:53.291347 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.277964    6133 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 21:54:53.291347 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.277966    6133 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 21:54:53.291347 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.277969    6133 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 21:54:53.291347 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.277972    6133 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 21:54:53.291347 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.277974    6133 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 21:54:53.291347 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.277977    6133 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 21:54:53.291347 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.277980    6133 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 21:54:53.291347 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.277983    6133 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 21:54:53.291347 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.277998    6133 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 21:54:53.291914 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.278002    6133 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 21:54:53.291914 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.278006    6133 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 21:54:53.291914 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.278011    6133 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 21:54:53.291914 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.278014    6133 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 21:54:53.291914 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.278019    6133 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 21:54:53.291914 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.278021    6133 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 21:54:53.291914 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.278024    6133 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 21:54:53.291914 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.278027    6133 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 21:54:53.291914 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.278029    6133 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 21:54:53.291914 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.278032    6133 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 21:54:53.291914 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.278034    6133 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 21:54:53.291914 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.278037    6133 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 21:54:53.291914 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.278041    6133 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 21:54:53.291914 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.278044    6133 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 21:54:53.291914 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.278047    6133 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 21:54:53.291914 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.278051    6133 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 21:54:53.291914 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.278054    6133 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 21:54:53.291914 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.278056    6133 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 21:54:53.291914 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.278059    6133 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 21:54:53.292439 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.278062    6133 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 21:54:53.292439 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.278065    6133 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 21:54:53.292439 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.278068    6133 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 21:54:53.292439 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.278071    6133 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 21:54:53.292439 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.278074    6133 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 21:54:53.292439 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.278076    6133 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 21:54:53.292439 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.278079    6133 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 21:54:53.292439 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.278081    6133 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 21:54:53.292439 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.278084    6133 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 21:54:53.292439 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.278086    6133 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 21:54:53.292439 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.278089    6133 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 21:54:53.292439 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.278091    6133 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 21:54:53.292439 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.278094    6133 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 21:54:53.292439 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.278097    6133 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 21:54:53.292439 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.278099    6133 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 21:54:53.292439 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.278102    6133 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 21:54:53.292439 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.278104    6133 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 21:54:53.292439 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.278108    6133 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 21:54:53.292439 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.278111    6133 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 21:54:53.292439 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.278113    6133 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 21:54:53.292931 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.278116    6133 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 21:54:53.292931 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.278119    6133 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 21:54:53.292931 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.278122    6133 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 21:54:53.292931 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.278124    6133 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 21:54:53.292931 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.278127    6133 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 21:54:53.292931 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.282627    6133 server.go:525] "Kubelet version" kubeletVersion="v1.35.9"
Oct 30 21:54:53.292931 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.282643    6133 server.go:527] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Oct 30 21:54:53.292931 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.282659    6133 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Oct 30 21:54:53.292931 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.282664    6133 watchdog_linux.go:138] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Oct 30 21:54:53.292931 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.282725    6133 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 21:54:53.292931 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.282732    6133 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 21:54:53.292931 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.282735    6133 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 21:54:53.292931 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.282738    6133 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 21:54:53.292931 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.282741    6133 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 21:54:53.292931 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.282744    6133 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 21:54:53.292931 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.282746    6133 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 21:54:53.292931 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.282749    6133 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 21:54:53.292931 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.282751    6133 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 21:54:53.292931 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.282754    6133 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 21:54:53.292931 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.282757    6133 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 21:54:53.293435 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.282760    6133 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 21:54:53.293435 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.282764    6133 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 21:54:53.293435 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.282769    6133 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 21:54:53.293435 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.282774    6133 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 21:54:53.293435 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.282778    6133 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 21:54:53.293435 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.282781    6133 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 21:54:53.293435 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.282784    6133 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 21:54:53.293435 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.282786    6133 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 21:54:53.293435 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.282789    6133 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 21:54:53.293435 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.282792    6133 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 21:54:53.293435 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.282794    6133 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 21:54:53.293435 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.282797    6133 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 21:54:53.293435 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.282799    6133 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 21:54:53.293435 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.282803    6133 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 21:54:53.293435 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.282808    6133 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 21:54:53.293435 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.282811    6133 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 21:54:53.293435 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.282815    6133 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 21:54:53.293435 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.282817    6133 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 21:54:53.293435 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.282820    6133 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 21:54:53.293904 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.282823    6133 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 21:54:53.293904 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.282825    6133 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 21:54:53.293904 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.282828    6133 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 21:54:53.293904 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.282831    6133 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 21:54:53.293904 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.282834    6133 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 21:54:53.293904 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.282836    6133 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 21:54:53.293904 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.282839    6133 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 21:54:53.293904 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.282844    6133 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 21:54:53.293904 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.282849    6133 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 21:54:53.293904 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.282853    6133 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 21:54:53.293904 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.282856    6133 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 21:54:53.293904 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.282859    6133 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 21:54:53.293904 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.282861    6133 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 21:54:53.293904 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.282864    6133 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 21:54:53.293904 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.282867    6133 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 21:54:53.293904 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.282869    6133 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 21:54:53.293904 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.282873    6133 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 21:54:53.293904 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.282875    6133 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 21:54:53.293904 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.282878    6133 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 21:54:53.294597 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.282881    6133 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 21:54:53.294597 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.282884    6133 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 21:54:53.294597 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.282887    6133 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 21:54:53.294597 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.282890    6133 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 21:54:53.294597 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.282893    6133 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 21:54:53.294597 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.282896    6133 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 21:54:53.294597 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.282898    6133 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 21:54:53.294597 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.282901    6133 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 21:54:53.294597 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.282904    6133 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 21:54:53.294597 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.282907    6133 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 21:54:53.294597 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.282909    6133 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 21:54:53.294597 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.282912    6133 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 21:54:53.294597 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.282915    6133 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 21:54:53.294597 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.282919    6133 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 21:54:53.294597 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.282923    6133 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 21:54:53.294597 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.282928    6133 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 21:54:53.294597 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.282931    6133 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 21:54:53.294597 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.282934    6133 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 21:54:53.294597 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.282937    6133 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 21:54:53.294597 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.282939    6133 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 21:54:53.295364 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.282942    6133 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 21:54:53.295364 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.282945    6133 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 21:54:53.295364 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.282948    6133 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 21:54:53.295364 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.282950    6133 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 21:54:53.295364 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.282953    6133 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 21:54:53.295364 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.282955    6133 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 21:54:53.295364 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.282959    6133 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 21:54:53.295364 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.282963    6133 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 21:54:53.295364 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.282966    6133 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 21:54:53.295364 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.282969    6133 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 21:54:53.295364 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.282973    6133 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 21:54:53.295364 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.282975    6133 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 21:54:53.295364 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.282978    6133 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 21:54:53.295364 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.282981    6133 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 21:54:53.295364 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.282983    6133 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 21:54:53.295364 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.282999    6133 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 21:54:53.295364 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.283004    6133 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 21:54:53.295364 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.283007    6133 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 21:54:53.295364 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.283010    6133 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 21:54:53.295961 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.283013    6133 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 21:54:53.295961 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.283016    6133 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 21:54:53.295961 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.283018    6133 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 21:54:53.295961 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.283021    6133 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 21:54:53.295961 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.283024    6133 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 21:54:53.295961 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.283026    6133 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 21:54:53.295961 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.283029    6133 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 21:54:53.295961 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.283031    6133 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 21:54:53.295961 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.283034    6133 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 21:54:53.295961 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.283037    6133 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 21:54:53.295961 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.283039    6133 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 21:54:53.295961 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.283042    6133 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 21:54:53.295961 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.283044    6133 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 21:54:53.295961 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.283047    6133 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 21:54:53.295961 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.283050    6133 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 21:54:53.295961 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.283052    6133 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 21:54:53.295961 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.283055    6133 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 21:54:53.295961 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.283057    6133 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 21:54:53.296599 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.283089    6133 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 30 21:54:53.296599 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.283241    6133 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 21:54:53.296599 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.283246    6133 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 21:54:53.296599 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.283250    6133 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 21:54:53.296599 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.283253    6133 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 21:54:53.296599 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.283256    6133 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 21:54:53.296599 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.283260    6133 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 21:54:53.296599 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.283263    6133 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 21:54:53.296599 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.283266    6133 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 21:54:53.296599 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.283269    6133 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 21:54:53.296599 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.283272    6133 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 21:54:53.296599 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.283275    6133 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 21:54:53.296599 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.283278    6133 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 21:54:53.296599 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.283281    6133 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 21:54:53.296599 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.283283    6133 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 21:54:53.296599 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.283286    6133 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 21:54:53.297023 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.283289    6133 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 21:54:53.297023 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.283291    6133 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 21:54:53.297023 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.283294    6133 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 21:54:53.297023 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.283297    6133 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 21:54:53.297023 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.283300    6133 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 21:54:53.297023 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.283302    6133 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 21:54:53.297023 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.283305    6133 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 21:54:53.297023 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.283307    6133 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 21:54:53.297023 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.283310    6133 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 21:54:53.297023 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.283312    6133 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 21:54:53.297023 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.283315    6133 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 21:54:53.297023 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.283318    6133 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 21:54:53.297023 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.283321    6133 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 21:54:53.297023 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.283323    6133 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 21:54:53.297023 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.283326    6133 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 21:54:53.297023 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.283329    6133 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 21:54:53.297023 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.283331    6133 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 21:54:53.297023 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.283334    6133 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 21:54:53.297023 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.283338    6133 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 21:54:53.297483 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.283342    6133 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 21:54:53.297483 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.283344    6133 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 21:54:53.297483 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.283347    6133 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 21:54:53.297483 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.283350    6133 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 21:54:53.297483 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.283353    6133 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 21:54:53.297483 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.283356    6133 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 21:54:53.297483 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.283358    6133 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 21:54:53.297483 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.283361    6133 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 21:54:53.297483 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.283364    6133 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 21:54:53.297483 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.283366    6133 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 21:54:53.297483 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.283369    6133 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 21:54:53.297483 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.283371    6133 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 21:54:53.297483 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.283374    6133 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 21:54:53.297483 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.283377    6133 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 21:54:53.297483 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.283380    6133 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 21:54:53.297483 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.283383    6133 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 21:54:53.297483 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.283385    6133 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 21:54:53.297483 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.283388    6133 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 21:54:53.297483 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.283390    6133 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 21:54:53.297483 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.283393    6133 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 21:54:53.297969 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.283395    6133 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 21:54:53.297969 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.283398    6133 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 21:54:53.297969 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.283401    6133 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 21:54:53.297969 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.283403    6133 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 21:54:53.297969 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.283406    6133 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 21:54:53.297969 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.283408    6133 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 21:54:53.297969 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.283411    6133 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 21:54:53.297969 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.283414    6133 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 21:54:53.297969 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.283417    6133 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 21:54:53.297969 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.283419    6133 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 21:54:53.297969 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.283422    6133 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 21:54:53.297969 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.283424    6133 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 21:54:53.297969 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.283427    6133 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 21:54:53.297969 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.283429    6133 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 21:54:53.297969 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.283432    6133 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 21:54:53.297969 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.283435    6133 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 21:54:53.297969 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.283437    6133 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 21:54:53.297969 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.283441    6133 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 21:54:53.297969 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.283443    6133 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 21:54:53.298444 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.283446    6133 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 21:54:53.298444 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.283449    6133 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 21:54:53.298444 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.283451    6133 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 21:54:53.298444 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.283454    6133 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 21:54:53.298444 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.283456    6133 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 21:54:53.298444 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.283461    6133 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 21:54:53.298444 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.283464    6133 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 21:54:53.298444 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.283467    6133 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 21:54:53.298444 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.283469    6133 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 21:54:53.298444 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.283472    6133 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 21:54:53.298444 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.283475    6133 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 21:54:53.298444 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.283477    6133 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 21:54:53.298444 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.283480    6133 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 21:54:53.298444 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.283483    6133 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 21:54:53.298444 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.283485    6133 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 21:54:53.298444 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.283489    6133 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 21:54:53.298444 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.283493    6133 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 21:54:53.298444 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.283497    6133 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 21:54:53.298444 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.283499    6133 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 21:54:53.298914 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.283514    6133 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 21:54:53.298914 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.283518    6133 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 21:54:53.298914 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.283523    6133 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 21:54:53.298914 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.283527    6133 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 21:54:53.298914 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.283530    6133 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 21:54:53.298914 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.283533    6133 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 21:54:53.298914 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.283535    6133 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 21:54:53.298914 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.283538    6133 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 21:54:53.298914 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.283541    6133 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 21:54:53.298914 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.283545    6133 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 21:54:53.298914 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.283548    6133 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 21:54:53.298914 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.283550    6133 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 21:54:53.298914 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.283554    6133 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 21:54:53.298914 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.283557    6133 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 21:54:53.298914 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.283582    6133 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 30 21:54:53.298914 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.283615    6133 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 21:54:53.299330 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.283619    6133 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 21:54:53.299330 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.283623    6133 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 21:54:53.299330 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.283626    6133 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 21:54:53.299330 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.283629    6133 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 21:54:53.299330 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.283631    6133 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 21:54:53.299330 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.283634    6133 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 21:54:53.299330 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.283636    6133 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 21:54:53.299330 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.283639    6133 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 21:54:53.299330 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.283642    6133 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 21:54:53.299330 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.283644    6133 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 21:54:53.299330 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.283647    6133 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 21:54:53.299330 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.283650    6133 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 21:54:53.299330 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.283652    6133 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 21:54:53.299330 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.283655    6133 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 21:54:53.299330 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.283658    6133 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 21:54:53.299330 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.283661    6133 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 21:54:53.299330 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.283663    6133 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 21:54:53.299330 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.283666    6133 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 21:54:53.299330 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.283669    6133 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 21:54:53.299790 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.283671    6133 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 21:54:53.299790 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.283674    6133 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 21:54:53.299790 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.283677    6133 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 21:54:53.299790 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.283679    6133 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 21:54:53.299790 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.283683    6133 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 21:54:53.299790 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.283687    6133 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 21:54:53.299790 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.283690    6133 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 21:54:53.299790 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.283696    6133 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 21:54:53.299790 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.283699    6133 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 21:54:53.299790 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.283702    6133 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 21:54:53.299790 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.283705    6133 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 21:54:53.299790 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.283707    6133 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 21:54:53.299790 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.283710    6133 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 21:54:53.299790 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.283713    6133 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 21:54:53.299790 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.283715    6133 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 21:54:53.299790 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.283718    6133 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 21:54:53.299790 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.283721    6133 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 21:54:53.299790 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.283723    6133 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 21:54:53.299790 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.283726    6133 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 21:54:53.300409 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.283729    6133 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 21:54:53.300409 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.283731    6133 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 21:54:53.300409 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.283734    6133 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 21:54:53.300409 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.283737    6133 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 21:54:53.300409 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.283739    6133 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 21:54:53.300409 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.283742    6133 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 21:54:53.300409 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.283744    6133 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 21:54:53.300409 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.283747    6133 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 21:54:53.300409 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.283750    6133 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 21:54:53.300409 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.283752    6133 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 21:54:53.300409 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.283755    6133 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 21:54:53.300409 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.283758    6133 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 21:54:53.300409 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.283760    6133 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 21:54:53.300409 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.283763    6133 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 21:54:53.300409 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.283766    6133 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 21:54:53.300409 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.283768    6133 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 21:54:53.300409 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.283771    6133 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 21:54:53.300409 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.283773    6133 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 21:54:53.300409 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.283776    6133 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 21:54:53.300882 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.283778    6133 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 21:54:53.300882 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.283783    6133 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 21:54:53.300882 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.283785    6133 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 21:54:53.300882 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.283788    6133 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 21:54:53.300882 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.283790    6133 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 21:54:53.300882 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.283794    6133 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 21:54:53.300882 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.283796    6133 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 21:54:53.300882 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.283799    6133 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 21:54:53.300882 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.283802    6133 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 21:54:53.300882 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.283804    6133 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 21:54:53.300882 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.283807    6133 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 21:54:53.300882 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.283810    6133 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 21:54:53.300882 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.283813    6133 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 21:54:53.300882 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.283815    6133 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 21:54:53.300882 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.283818    6133 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 21:54:53.300882 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.283821    6133 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 21:54:53.300882 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.283824    6133 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 21:54:53.300882 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.283826    6133 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 21:54:53.300882 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.283829    6133 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 21:54:53.300882 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.283832    6133 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 21:54:53.301381 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.283834    6133 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 21:54:53.301381 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.283838    6133 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 21:54:53.301381 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.283842    6133 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 21:54:53.301381 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.283845    6133 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 21:54:53.301381 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.283848    6133 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 21:54:53.301381 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.283851    6133 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 21:54:53.301381 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.283853    6133 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 21:54:53.301381 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.283856    6133 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 21:54:53.301381 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.283858    6133 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 21:54:53.301381 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.283861    6133 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 21:54:53.301381 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.283864    6133 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 21:54:53.301381 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.283866    6133 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 21:54:53.301381 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.283869    6133 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 21:54:53.301381 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.283874    6133 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 21:54:53.301381 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.283877    6133 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 21:54:53.301381 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.283879    6133 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 21:54:53.301381 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.283882    6133 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 21:54:53.301381 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.283884    6133 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 21:54:53.301381 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.283888    6133 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 21:54:53.301874 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.283890    6133 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 21:54:53.301874 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.283893    6133 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 21:54:53.301874 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.283896    6133 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 21:54:53.301874 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.283899    6133 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 21:54:53.301874 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.283902    6133 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 21:54:53.301874 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.283905    6133 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 21:54:53.301874 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.283907    6133 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 21:54:53.301874 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.283910    6133 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 21:54:53.301874 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:54:53.283912    6133 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 21:54:53.301874 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.283969    6133 server.go:957] "Client rotation is on, will bootstrap in background"
Oct 30 21:54:53.301874 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.284978    6133 bootstrap.go:85] "Current kubeconfig file contents are still valid, no bootstrap necessary"
Oct 30 21:54:53.301874 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.285051    6133 certificate_store.go:147] "Loading cert/key pair from a file" filePath="/var/lib/kubelet/pki/kubelet-client-current.pem"
Oct 30 21:54:53.301874 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.285239    6133 server.go:1014] "Starting client certificate rotation"
Oct 30 21:54:53.301874 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.285433    6133 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Oct 30 21:54:53.301874 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.285518    6133 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-10-29 21:48:42 +0000 UTC" deadline="2028-06-21 23:18:29.610721844 +0000 UTC"
Oct 30 21:54:53.301874 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.285580    6133 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14401h23m36.325143909s"
Oct 30 21:54:53.302292 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.285717    6133 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Oct 30 21:54:53.302292 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.286781    6133 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Oct 30 21:54:53.302292 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.288595    6133 log.go:25] "Validated CRI v1 runtime API"
Oct 30 21:54:53.302292 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.290865    6133 log.go:25] "Validated CRI v1 image API"
Oct 30 21:54:53.302292 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.291599    6133 server.go:1424] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Oct 30 21:54:53.302292 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.293713    6133 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 e7f760cd-c531-4205-a373-dccc3d16141d:/dev/nvme0n1p3 ede47d30-7089-4072-bf4e-c4f450f09bf8:/dev/nvme0n1p4]
Oct 30 21:54:53.302536 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.293734    6133 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:24 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:26 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/032e17cbfd6990eb3c9bc358001153b8ae4e45b54c916d214281a29bb48fe869/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/032e17cbfd6990eb3c9bc358001153b8ae4e45b54c916d214281a29bb48fe869/userdata/shm major:0 minor:68 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/0bc4cf08b77946fc12c13d79e800268cebb120d1de733fa971a67b52e17c9336/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/0bc4cf08b77946fc12c13d79e800268cebb120d1de733fa971a67b52e17c9336/userdata/shm major:0 minor:421 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/210e318062886f16a6ccfb46ef82db381f07dd4d7d0518075c707f0853ed9ea9/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/210e318062886f16a6ccfb46ef82db381f07dd4d7d0518075c707f0853ed9ea9/userdata/shm major:0 minor:90 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/3cf409dd46f74bbfa1745012a0345fc7230a78cf391809e4d473e6cf35376e9e/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/3cf409dd46f74bbfa1745012a0345fc7230a78cf391809e4d473e6cf35376e9e/userdata/shm major:0 minor:94 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/4397165a20a420aac8ad6d0f822e75d8972b23e73db86d9ba549c829116c4480/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/4397165a20a420aac8ad6d0f822e75d8972b23e73db86d9ba549c829116c4480/userdata/shm major:0 minor:419 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/4efea9f6053c04c56805d6877572edc5c4aa453e6c534f35eefcdfdf5cc9947b/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/4efea9f6053c04c56805d6877572edc5c4aa453e6c534f35eefcdfdf5cc9947b/userdata/shm major:0 minor:72 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/539e430a94bd9fa3fb11634f988f610e9eced549a966b67a5185fe5759aad92c/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/539e430a94bd9fa3fb11634f988f610e9eced549a966b67a5185fe5759aad92c/userdata/shm major:0 minor:98 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/81edfd255828d18cc1f4fca31e927255f649b6b1bcbba23bb65a1819dd9a9b65/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/81edfd255828d18cc1f4fca31e927255f649b6b1bcbba23bb65a1819dd9a9b65/userdata/shm major:0 minor:246 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/83f5c0d6735a2e35b4e005074f2c7f583f92ea7a94d680c9c430b8d1df568d4c/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/83f5c0d6735a2e35b4e005074f2c7f583f92ea7a94d680c9c430b8d1df568d4c/userdata/shm major:0 minor:379 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/8ce50405433f59d6286d2bbf2f1008372c665afdfb13349ef7546cf0b1be08a4/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/8ce50405433f59d6286d2bbf2f1008372c665afdfb13349ef7546cf0b1be08a4/userdata/shm major:0 minor:205 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/9bd7faef61deec19be60215e6651eb7da94b352c4e82286d682c30c02b91f953/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/9bd7faef61deec19be60215e6651eb7da94b352c4e82286d682c30c02b91f953/userdata/shm major:0 minor:417 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/9ce524660d905f015a7f4f67394797c49fed963f767dec7a56702ccf03654c8b/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/9ce524660d905f015a7f4f67394797c49fed963f767dec7a56702ccf03654c8b/userdata/shm major:0 minor:76 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/a1a23f1cc1936fcf6f0fbc3af4009a0cd4a17790ed28b13c1dfc5accffe8f597/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/a1a23f1cc1936fcf6f0fbc3af4009a0cd4a17790ed28b13c1dfc5accffe8f597/userdata/shm major:0 minor:47 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/a3f130547617214efc2a1c06555379109c660bad2501f887be595e3d279ea8a4/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/a3f130547617214efc2a1c06555379109c660bad2501f887be595e3d279ea8a4/userdata/shm major:0 minor:49 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/bfb89131d151d9c07894bf934194c2ebb93199fbe0f6297f748d4cdf54d68975/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/bfb89131d151d9c07894bf934194c2ebb93199fbe0f6297f748d4cdf54d68975/userdata/shm major:0 minor:320 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/c0fd54d2f9eade40d52d73aa5fd6dfb74c9a1ed40c855e3e0773cb680ead5d10/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/c0fd54d2f9eade40d52d73aa5fd6dfb74c9a1ed40c855e3e0773cb680ead5d10/userdata/shm major:0 minor:207 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/d65ef881919ac2a78210156713c755f348c9da7cd09bd6af266bbdfa0cb76498/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/d65ef881919ac2a78210156713c755f348c9da7cd09bd6af266bbdfa0cb76498/userdata/shm major:0 minor:326 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/e87446335be5893b70428a03a70ad83dac422290b1eff305dbe509ee7e35170d/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/e87446335be5893b70428a03a70ad83dac422290b1eff305dbe509ee7e35170d/userdata/shm major:0 minor:262 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/e963d0a97dfea14806021bbebf9320c7147bdee479c686f3a196b1ca0ed1b577/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/e963d0a97dfea14806021bbebf9320c7147bdee479c686f3a196b1ca0ed1b577/userdata/shm major:0 minor:80 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/ea0896e86f6df5158e1a1ecc5b41aa8e849df46707c6cc80f76940fc2dadcd09/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/ea0896e86f6df5158e1a1ecc5b41aa8e849df46707c6cc80f76940fc2dadcd09/userdata/shm major:0 minor:209 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/ee565a8a0022cec61d84d7110490393c151780d6fd4c9eb513e311b6a19048d8/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/ee565a8a0022cec61d84d7110490393c151780d6fd4c9eb513e311b6a19048d8/userdata/shm major:0 minor:86 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/efaaacbcd02ded84d312f70628a2cf5b042af9118274819e3ab06919565b3f46/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/efaaacbcd02ded84d312f70628a2cf5b042af9118274819e3ab06919565b3f46/userdata/shm major:0 minor:252 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:32 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/15e1de31-92a1-4179-8ec1-400346880d1d/volumes/kubernetes.io~projected/kube-api-access-lgk4w:{mountpoint:/var/lib/kubelet/pods/15e1de31-92a1-4179-8ec1-400346880d1d/volumes/kubernetes.io~projected/kube-api-access-lgk4w major:0 minor:64 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/17da628d-31f5-456c-ad37-52c884d22cf4/volumes/kubernetes.io~projected/kube-api-access-bw6lg:{mountpoint:/var/lib/kubelet/pods/17da628d-31f5-456c-ad37-52c884d22cf4/volumes/kubernetes.io~projected/kube-api-access-bw6lg major:0 minor:248 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/17da628d-31f5-456c-ad37-52c884d22cf4/volumes/kubernetes.io~secret/kube-state-metrics-kube-rbac-proxy-config:{mountpoint:/var/lib/kubelet/pods/17da628d-31f5-456c-ad37-52c884d22cf4/volumes/kubernetes.io~secret/kube-state-metrics-kube-rbac-proxy-config major:0 minor:244 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/17da628d-31f5-456c-ad37-52c884d22cf4/volumes/kubernetes.io~secret/kube-state-metrics-tls:{mountpoint:/var/lib/kubelet/pods/17da628d-31f5-456c-ad37-52c884d22cf4/volumes/kubernetes.io~secret/kube-state-metrics-tls major:0 minor:245 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/209b4e58-e6c2-48f1-8589-f7b6be79720b/volumes/kubernetes.io~projected/kube-api-access-mrtmr:{mountpoint:/var/lib/kubelet/pods/209b4e58-e6c2-48f1-8589-f7b6be79720b/volumes/kubernetes.io~projected/kube-api-access-mrtmr major:0 minor:202 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/209b4e58-e6c2-48f1-8589-f7b6be79720b/volumes/kubernetes.io~secret/metrics-tls:{mountpoint:/var/lib/kubelet/pods/209b4e58-e6c2-48f1-8589-f7b6be79720b/volumes/kubernetes.io~secret/metrics-tls major:0 minor:197 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/2bcee03a-9b40-4155-8d81-43dece439162/volumes/kubernetes.io~secret/agent-certs:{mountpoint:/var/lib/kubelet/pods/2bcee03a-9b40-4155-8d81-43dece439162/volumes/kubernetes.io~secret/agent-certs major:0 minor:53 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/32ae0d42-4b76-47b7-b9d6-26b09f00ba5d/volumes/kubernetes.io~projected/kube-api-access-ljsqq:{mountpoint:/var/lib/kubelet/pods/32ae0d42-4b76-47b7-b9d6-26b09f00ba5d/volumes/kubernetes.io~projected/kube-api-access-ljsqq major:0 minor:416 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/3e60fceb-8682-45cc-a472-f3d7017c3395/volumes/kubernetes.io~projected/kube-api-access-6bsvp:{mountpoint:/var/lib/kubelet/pods/3e60fceb-8682-45cc-a472-f3d7017c3395/volumes/kubernetes.io~projected/kube-api-access-6bsvp major:0 minor:61 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/446beacf-70a0-4d47-96ea-bbe1ca4a5f28/volume-subpaths/run-systemd/ovnkube-controller/6:{mountpoint:/var/lib/kubelet/pods/446beacf-70a0-4d47-96ea-bbe1ca4a5f28/volume-subpaths/run-systemd/ovnkube-controller/6 major:0 minor:26 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/446beacf-70a0-4d47-96ea-bbe1ca4a5f28/volumes/kubernetes.io~projected/kube-api-access-mkrsw:{mountpoint:/var/lib/kubelet/pods/446beacf-70a0-4d47-96ea-bbe1ca4a5f28/volumes/kubernetes.io~projected/kube-api-access-mkrsw major:0 minor:66 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/446beacf-70a0-4d47-96ea-bbe1ca4a5f28/volumes/kubernetes.io~secret/ovn-node-metrics-cert:{mountpoint:/var/lib/kubelet/pods/446beacf-70a0-4d47-96ea-bbe1ca4a5f28/volumes/kubernetes.io~secret/ovn-node-metrics-cert major:0 minor:55 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/4621048c-230e-441f-85fc-423e4ba525ec/volumes/kubernetes.io~projected/kube-api-access-9f2kg:{mountpoint:/var/lib/kubelet/pods/4621048c-230e-441f-85fc-423e4ba525ec/volumes/kubernetes.io~projected/kube-api-access-9f2kg major:0 minor:204 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/4621048c-230e-441f-85fc-423e4ba525ec/volumes/kubernetes.io~secret/cert:{mountpoint:/var/lib/kubelet/pods/4621048c-230e-441f-85fc-423e4ba525ec/volumes/kubernetes.io~secret/cert major:0 minor:203 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/484e37a0-fd1d-4fd0-a200-939b8945984b/volumes/kubernetes.io~projected/kube-api-access-n87x8:{mountpoint:/var/lib/kubelet/pods/484e37a0-fd1d-4fd0-a200-939b8945984b/volumes/kubernetes.io~projected/kube-api-access-n87x8 major:0 minor:241 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/484e37a0-fd1d-4fd0-a200-939b8945984b/volumes/kubernetes.io~secret/openshift-state-metrics-kube-rbac-proxy-config:{mountpoint:/var/lib/kubelet/pods/484e37a0-fd1d-4fd0-a200-939b8945984b/volumes/kubernetes.io~secret/openshift-state-metrics-kube-rbac-proxy-config major:0 minor:236 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/484e37a0-fd1d-4fd0-a200-939b8945984b/volumes/kubernetes.io~secret/openshift-state-metrics-tls:{mountpoint:/var/lib/kubelet/pods/484e37a0-fd1d-4fd0-a200-939b8945984b/volumes/kubernetes.io~secret/openshift-state-metrics-tls major:0 minor:240 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/532658b6-c86e-4feb-89cb-14a91316a7a0/volumes/kubernetes.io~projected/kube-api-access-r6wtm:{mountpoint:/var/lib/kubelet/pods/532658b6-c86e-4feb-89cb-14a91316a7a0/volumes/kubernetes.io~projected/kube-api-access-r6wtm major:0 minor:59 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/532658b6-c86e-4feb-89cb-14a91316a7a0/volumes/kubernetes.io~secret/metrics-certs:{mountpoint:/var/lib/kubelet/pods/532658b6-c86e-4feb-89cb-14a91316a7a0/volumes/kubernetes.io~secret/metrics-certs major:0 minor:415 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/5d5057fc-dd8a-4dce-b8bb-70b2dc3b074c/volumes/kubernetes.io~projected/kube-api-access-kncjh:{mountpoint:/var/lib/kubelet/pods/5d5057fc-dd8a-4dce-b8bb-70b2dc3b074c/volumes/kubernetes.io~projected/kube-api-access-kncjh major:0 minor:62 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/613f514c-6011-4936-8df5-a59b960cd3a6/volumes/kubernetes.io~projected/kube-api-access-h6b6r:{mountpoint:/var/lib/kubelet/pods/613f514c-6011-4936-8df5-a59b960cd3a6/volumes/kubernetes.io~projected/kube-api-access-h6b6r major:0 minor:60 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/6192d2be-bba7-4099-b54c-e7e9b8d75389/volumes/kubernetes.io~projected/kube-api-access-jw4wk:{mountpoint:/var/lib/kubelet/pods/6192d2be-bba7-4099-b54c-e7e9b8d75389/volumes/kubernetes.io~projected/kube-api-access-jw4wk major:0 minor:201 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/6192d2be-bba7-4099-b54c-e7e9b8d75389/volumes/kubernetes.io~secret/insights-runtime-extractor-tls:{mountpoint:/var/lib/kubelet/pods/6192d2be-bba7-4099-b54c-e7e9b8d75389/volumes/kubernetes.io~secret/insights-runtime-extractor-tls major:0 minor:45 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/7ddfa67d-20a0-473d-937f-25fd8e0ba80e/volumes/kubernetes.io~projected/kube-api-access-wsnr5:{mountpoint:/var/lib/kubelet/pods/7ddfa67d-20a0-473d-937f-25fd8e0ba80e/volumes/kubernetes.io~projected/kube-api-access-wsnr5 major:0 minor:249 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/7ddfa67d-20a0-473d-937f-25fd8e0ba80e/volumes/kubernetes.io~secret/node-exporter-kube-rbac-proxy-config:{mountpoint:/var/lib/kubelet/pods/7ddfa67d-20a0-473d-937f-25fd8e0ba80e/volumes/kubernetes.io~secret/node-exporter-kube-rbac-proxy-config major:0 minor:242 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/7ddfa67d-20a0-473d-937f-25fd8e0ba80e/volumes/kubernetes.io~secret/node-exporter-tls:{mountpoint:/var/lib/kubelet/pods/7ddfa67d-20a0-473d-937f-25fd8e0ba80e/volumes/kubernetes.io~secret/node-exporter-tls major:0 minor:243 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/9d2cced5-9ceb-421f-b831-8370a5ee0ca9/volumes/kubernetes.io~projected/kube-api-access-w78hr:{mountpoint:/var/lib/kubelet/pods/9d2cced5-9ceb-421f-b831-8370a5ee0ca9/volumes/kubernetes.io~projected/kube-api-access-w78hr major:0 minor:325 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/9d2cced5-9ceb-421f-b831-8370a5ee0ca9/volumes/kubernetes.io~secret/monitoring-plugin-cert:{mountpoint:/var/lib/kubelet/pods/9d2cced5-9ceb-421f-b831-8370a5ee0ca9/volumes/kubernetes.io~secret/monitoring-plugin-cert major:0 minor:324 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/af83b188-09cb-4621-a9da-be33856d19f0/volumes/kubernetes.io~secret/original-pull-secret:{mountpoint:/var/lib/kubelet/pods/af83b188-09cb-4621-a9da-be33856d19f0/volumes/kubernetes.io~secret/original-pull-secret major:0 minor:411 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/bd93518f-2f96-4483-9e92-513109dd37cb/volume-subpaths/web-config/config-reloader/2:{mountpoint:/var/lib/kubelet/pods/bd93518f-2f96-4483-9e92-513109dd37cb/volume-subpaths/web-config/config-reloader/2 major:0 minor:374 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/bd93518f-2f96-4483-9e92-513109dd37cb/volume-subpaths/web-config/init-config-reloader/2:{mountpoint:/var/lib/kubelet/pods/bd93518f-2f96-4483-9e92-513109dd37cb/volume-subpaths/web-config/init-config-reloader/2 major:0 minor:374 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/bd93518f-2f96-4483-9e92-513109dd37cb/volume-subpaths/web-config/prometheus/15:{mountpoint:/var/lib/kubelet/pods/bd93518f-2f96-4483-9e92-513109dd37cb/volume-subpaths/web-config/prometheus/15 major:0 minor:374 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/bd93518f-2f96-4483-9e92-513109dd37cb/volumes/kubernetes.io~empty-dir/config-out:{mountpoint:/var/lib/kubelet/pods/bd93518f-2f96-4483-9e92-513109dd37cb/volumes/kubernetes.io~empty-dir/config-out major:0 minor:376 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/bd93518f-2f96-4483-9e92-513109dd37cb/volumes/kubernetes.io~projected/kube-api-access-qrt69:{mountpoint:/var/lib/kubelet/pods/bd93518f-2f96-4483-9e92-513109dd37cb/volumes/kubernetes.io~projected/kube-api-access-qrt69 major:0 minor:378 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/bd93518f-2f96-4483-9e92-513109dd37cb/volumes/kubernetes.io~projected/tls-assets:{mountpoint:/var/lib/kubelet/pods/bd93518f-2f96-4483-9e92-513109dd37cb/volumes/kubernetes.io~projected/tls-assets major:0 minor:375 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/bd93518f-2f96-4483-9e92-513109dd37cb/volumes/kubernetes.io~secret/config:{mountpoint:/var/lib/kubelet/pods/bd93518f-2f96-4483-9e92-513109dd37cb/volumes/kubernetes.io~secret/config major:0 minor:363 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/bd93518f-2f96-4483-9e92-513109dd37cb/volumes/kubernetes.io~secret/secret-grpc-tls:{mountpoint:/var/lib/kubelet/pods/bd93518f-2f96-4483-9e92-513109dd37cb/volumes/kubernetes.io~secret/secret-grpc-tls major:0 minor:377 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/bd93518f-2f96-4483-9e92-513109dd37cb/volumes/kubernetes.io~secret/secret-kube-rbac-proxy:{mountpoint:/var/lib/kubelet/pods/bd93518f-2f96-4483-9e92-513109dd37cb/volumes/kubernetes.io~secret/secret-kube-rbac-proxy major:0 minor:370 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/bd93518f-2f96-4483-9e92-513109dd37cb/volumes/kubernetes.io~secret/secret-metrics-client-certs:{mountpoint:/var/lib/kubelet/pods/bd93518f-2f96-4483-9e92-513109dd37cb/volumes/kubernetes.io~secret/secret-metrics-client-certs major:0 minor:373 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/bd93518f-2f96-4483-9e92-513109dd37cb/volumes/kubernetes.io~secret/secret-prometheus-k8s-kube-rbac-proxy-web:{mountpoint:/var/lib/kubelet/pods/bd93518f-2f96-4483-9e92-513109dd37cb/volumes/kubernetes.io~secret/secret-prometheus-k8s-kube-rbac-proxy-web major:0 minor:369 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/bd93518f-2f96-4483-9e92-513109dd37cb/volumes/kubernetes.io~secret/secret-prometheus-k8s-thanos-sidecar-tls:{mountpoint:/var/lib/kubelet/pods/bd93518f-2f96-4483-9e92-513109dd37cb/volumes/kubernetes.io~secret/secret-prometheus-k8s-thanos-sidecar-tls major:0 minor:371 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/bd93518f-2f96-4483-9e92-513109dd37cb/volumes/kubernetes.io~secret/secret-prometheus-k8s-tls:{mountpoint:/var/lib/kubelet/pods/bd93518f-2f96-4483-9e92-513109dd37cb/volumes/kubernetes.io~secret/secret-prometheus-k8s-tls major:0 minor:368 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/bd93518f-2f96-4483-9e92-513109dd37cb/volumes/kubernetes.io~secret/thanos-prometheus-http-client-file:{mountpoint:/var/lib/kubelet/pods/bd93518f-2f96-4483-9e92-513109dd37cb/volumes/kubernetes.io~secret/thanos-prometheus-http-client-file major:0 minor:372 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/bd93518f-2f96-4483-9e92-513109dd37cb/volumes/kubernetes.io~secret/web-config:{mountpoint:/var/lib/kubelet/pods/bd93518f-2f96-4483-9e92-513109dd37cb/volumes/kubernetes.io~secret/web-config major:0 minor:374 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/bebdb584-1a96-453d-8a6d-b332e47bec3d/volumes/kubernetes.io~empty-dir/etc-tuned:{mountpoint:/var/lib/kubelet/pods/bebdb584-1a96-453d-8a6d-b332e47bec3d/volumes/kubernetes.io~empty-dir/etc-tuned major:0 minor:54 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/bebdb584-1a96-453d-8a6d-b332e47bec3d/volumes/kubernetes.io~empty-dir/tmp:{mountpoint:/var/lib/kubelet/pods/bebdb584-1a96-453d-8a6d-b332e47bec3d/volumes/kubernetes.io~empty-dir/tmp major:0 minor:56 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/bebdb584-1a96-453d-8a6d-b332e47bec3d/volumes/kubernetes.io~projected/kube-api-access-r4qz4:{mountpoint:/var/lib/kubelet/pods/bebdb584-1a96-453d-8a6d-b332e47bec3d/volumes/kubernetes.io~projected/kube-api-access-r4qz4 major:0 minor:63 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/fbd8f09b-9962-4910-8b25-267cbbb3ea5d/volumes/kubernetes.io~projected/kube-api-access-zzw6w:{mountpoint:/var/lib/kubelet/pods/fbd8f09b-9962-4910-8b25-267cbbb3ea5d/volumes/kubernetes.io~projected/kube-api-access-zzw6w major:0 minor:319 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/fbd8f09b-9962-4910-8b25-267cbbb3ea5d/volumes/kubernetes.io~secret/secret-grpc-tls:{mountpoint:/var/lib/kubelet/pods/fbd8f09b-9962-4910-8b25-267cbbb3ea5d/volumes/kubernetes.io~secret/secret-grpc-tls major:0 minor:314 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/fbd8f09b-9962-4910-8b25-267cbbb3ea5d/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy:{mountpoint:/var/lib/kubelet/pods/fbd8f09b-9962-4910-8b25-267cbbb3ea5d/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy major:0 minor:317 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/fbd8f09b-9962-4910-8b25-267cbbb3ea5d/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy-metrics:{mountpoint:/var/lib/kubelet/pods/fbd8f09b-9962-4910-8b25-267cbbb3ea5d/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy-metrics major:0 minor:316 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/fbd8f09b-9962-4910-8b25-267cbbb3ea5d/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy-rules:{mountpoint:/var/lib/kubelet/pods/fbd8f09b-9962-4910-8b25-267cbbb3ea5d/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy-rules major:0 minor:318 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/fbd8f09b-9962-4910-8b25-267cbbb3ea5d/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy-web:{mountpoint:/var/lib/kubelet/pods/fbd8f09b-9962-4910-8b25-267cbbb3ea5d/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy-web major:0 minor:310 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/fbd8f09b-9962-4910-8b25-267cbbb3ea5d/volumes/kubernetes.io~secret/secret-thanos-querier-tls:{mountpoint:/var/lib/kubelet/pods/fbd8f09b-9962-4910-8b25-267cbbb3ea5d/volumes/kubernetes.io~secret/secret-thanos-querier-tls major:0 minor:315 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/fc4a67d9-8c7c-4f71-9a8a-2bba09329bc6/volumes/kubernetes.io~projected/kube-api-access-jp25c:{mountpoint:/var/lib/kubelet/pods/fc4a67d9-8c7c-4f71-9a8a-2bba09329bc6/volumes/kubernetes.io~projected/kube-api-access-jp25c major:0 minor:67 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/fe472529-1294-46a1-a666-dc00d5d37eca/volumes/kubernetes.io~projected/kube-api-access-nbqv6:{mountpoint:/var/lib/kubelet/pods/fe472529-1294-46a1-a666-dc00d5d37eca/volumes/kubernetes.io~projected/kube-api-access-nbqv6 major:0 minor:65 fsType:tmpfs blockSize:0} composefs_0-34:{mountpoint:/ major:0 minor:34 fsType:overlay blockSize:0} overlay_0-100:{mountpoint:/var/lib/containers/storage/overlay/48d6762df2d56c80f9f18ffb421ef309de35ee5b397e4960fd677be822817b40/merged major:0 minor:100 fsType:overlay blockSize:0} overlay_0-102:{mountpoint:/var/lib/containers/storage/overlay/50f539dd11cd0b4cb88408f53b154a981e09de45fd897b3269e4ea795de8db29/merged major:0 minor:102 fsType:overlay blockSize:0} overlay_0-104:{mountpoint:/var/lib/containers/storage/overlay/2aa7183e6213c4fae1ef415667bc3cbccb44646963c07bfb73591d4df532f928/merged major:0 minor:104 fsType:overlay blockSize:0} overlay_0-109:{mountpoint:/var/lib/containers/storage/overlay/fc8d6e31c90d6af4782ef7025e4cd8cae02b61fbceed330673a95e95f6f86c6f/merged major:0 minor:109 fsType:overlay blockSize:0} overlay_0-114:{mountpoint:/var/lib/containers/storage/overlay/cdb32607f47946e1e65721cb1156fff02076878ee951307f668b47a19e44c28f/merged major:0 minor:114 fsType:overlay blockSize:0} overlay_0-119:{mountpoint:/var/lib/containers/storage/overlay/95e955507b8658cdf7d97b4ddea6a3c203debdd01e39b7ae07d972415fde2730/merged major:0 minor:119 fsType:overlay blockSize:0} overlay_0-124:{mountpoint:/var/lib/containers/storage/overlay/262bdbd7b63c89223424114452646f09d1646c9360c9839ef0e539737e9b1010/merged major:0 minor:124 fsType:overlay blockSize:0} overlay_0-127:{mountpoint:/var/lib/containers/storage/overlay/e824da7e304fe3638e8a71b0a1bdc209e2dff0dc64a140d8d82d6b20354f3a96/merged major:0 minor:127 fsType:overlay blockSize:0} overlay_0-132:{mountpoint:/var/lib/containers/storage/overlay/653d7d0a5f688440203d59e19377ee459f362119cbf308816f4ead620533b383/merged major:0 minor:132 fsType:overlay blockSize:0} overlay_0-137:{mountpoint:/var/lib/containers/storage/overlay/b613deb2f03d7afcc9bac2ca4540315687961a7689ab121a011714225d7f93fc/merged major:0 minor:137 fsType:overlay blockSize:0} overlay_0-139:{mountpoint:/var/lib/containers/storage/overlay/141f39492294f0cd3e0ab571fe67433e66d8d2b6b7d1b36756258702e2920a90/merged major:0 minor:139 fsType:overlay blockSize:0} overlay_0-143:{mountpoint:/var/lib/containers/storage/overlay/711c1ae7816815f1eb584954289fc093956ebb1a4c32b96346327753606e6630/merged major:0 minor:143 fsType:overlay blockSize:0} overlay_0-146:{mountpoint:/var/lib/containers/storage/overlay/aa3aea41309be47a49a743e6837fbd8a8d9f9fc1647025659fd67a28b074f319/merged major:0 minor:146 fsType:overlay blockSize:0} overlay_0-157:{mountpoint:/var/lib/containers/storage/overlay/1303a77f68bdf790a390fab939b8623c17a9519785f62743c8b4b0cd7e3e07e6/merged major:0 minor:157 fsType:overlay blockSize:0} overlay_0-162:{mountpoint:/var/lib/containers/storage/overlay/07cfc03e680bbbe81e1706f7e51998e1c7b4280e719ad8203e2df0421f24f6d0/merged major:0 minor:162 fsType:overlay blockSize:0} overlay_0-164:{mountpoint:/var/lib/containers/storage/overlay/567cd2d728618e20bfbdc0256579c52638dfc033614a63673c2cd922fffca60b/merged major:0 minor:164 fsType:overlay blockSize:0} overlay_0-169:{mountpoint:/var/lib/containers/storage/overlay/b5d827ac9dfac2162d355b5d1483b7db9d3d945129232c5d05ab596eec13a272/merged major:0 minor:169 fsType:overlay blockSize:0} overlay_0-177:{mountpoint:/var/lib/containers/storage/overlay/68f2785c94bb9a1011aea589037a6baf0147adfa3046c1301c4cd061f7830a4c/merged major:0 minor:177 fsType:overlay blockSize:0} overlay_0-182:{mountpoint:/var/lib/containers/storage/overlay/9d5534ef392e35a3a555435f805044c1ed7eb44f77066bea9c63a824dcb435fa/merged major:0 minor:182 fsType:overlay blockSize:0} overlay_0-187:{mountpoint:/var/lib/containers/storage/overlay/de7c8b796338554b2695c7469a9de3ffe2b1cd0175cfdd8748a2a675509d7967/merged major:0 minor:187 fsType:overlay blockSize:0} overlay_0-192:{mountpoint:/var/lib/containers/storage/overlay/d1927e879a9119225dfe8ec5d0b34119da8783e5e0ed569216b6bffc5f7f6f55/merged major:0 minor:192 fsType:overlay blockSize:0} overlay_0-211:{mountpoint:/var/lib/containers/storage/overlay/f1949ee0f73c278d81d588f5fa2e9279603e99519a85634c22660be20ea96d95/merged major:0 minor:211 fsType:overlay blockSize:0} overlay_0-213:{mountpoint:/var/lib/containers/storage/overlay/373b63019fb3e280234b74cebc18535425c2ba71cb0b9e1f7fcac8ed0cba8f2f/merged major:0 minor:213 fsType:overlay blockSize:0} overlay_0-215:{mountpoint:/var/lib/containers/storage/overlay/e357656d875452f34c9f2390e598753b2c3f78160c301311ac7f5313822e7fdd/merged major:0 minor:215 fsType:overlay blockSize:0} overlay_0-217:{mountpoint:/var/lib/containers/storage/overlay/6222ce17363a61382f42469f7f8864cd8ea704ebb6740d39673f4a12abe125d4/merged major:0 minor:217 fsType:overlay blockSize:0} overlay_0-219:{mountpoint:/var/lib/containers/storage/overlay/2d47721f276cd18979ce2891df7385939151db2b5993cb3b51081bc9d7cd1d7a/merged major:0 minor:219 fsType:overlay blockSize:0} overlay_0-225:{mountpoint:/var/lib/containers/storage/overlay/199b7770b068860e4cfafac35ef249caf47f4cdfcccfddbe64ab1a1a83d5c638/merged major:0 minor:225 fsType:overlay blockSize:0} overlay_0-230:{mountpoint:/var/lib/containers/storage/overlay/9573553ca9560480fb07047cdb8068d2e436eb3f18a688e06678419ba35b8d31/merged major:0 minor:230 fsType:overlay blockSize:0} overlay_0-250:{mountpoint:/var/lib/containers/storage/overlay/21a4850a1477f86a3e0c87ad5b57f63f075f44479f10b94b9b90cef24284be74/merged major:0 minor:250 fsType:overlay blockSize:0} overlay_0-258:{mountpoint:/var/lib/containers/storage/overlay/369c3aaa85ddf2e79fcfdeda55d5ca5f3dfe5c881da98e34b3f03fdbeafa106c/merged major:0 minor:258 fsType:overlay blockSize:0} overlay_0-260:{mountpoint:/var/lib/containers/storage/overlay/1ae6a1cce33536959010dd37998fa873786c04494eb11cc162a30c33afb30ba2/merged major:0 minor:260 fsType:overlay blockSize:0} overlay_0-264:{mountpoint:/var/lib/containers/storage/overlay/8d33d138a988f42917f437906e8442f679c205da5a29b0c1ca69a01403d26c98/merged major:0 minor:264 fsType:overlay blockSize:0} overlay_0-269:{mountpoint:/var/lib/containers/storage/overlay/493ac40158013954f91bb4b12e1e4d48c3cce70ab575edd0db2498c880063a67/merged major:0 minor:269 fsType:overlay blockSize:0} overlay_0-271:{mountpoint:/var/lib/containers/storage/overlay/6e7d077b68116b1a549ae99ffbc9fb4ef70a7f0916430747af7c8b76a31143b4/merged major:0 minor:271 fsType:overlay blockSize:0} overlay_0-277:{mountpoint:/var/lib/containers/storage/overlay/5aacd4bd632f9c025b46e93a2ecf1ab49882c8af041f264be57e9fd1a806130b/merged major:0 minor:277 fsType:overlay blockSize:0} overlay_0-282:{mountpoint:/var/lib/containers/storage/overlay/44f73d4ce8b60514e97eba2e217e046842c19a65d9aa7736f35ee1a34004bfe5/merged major:0 minor:282 fsType:overlay blockSize:0} overlay_0-284:{mountpoint:/var/lib/containers/storage/overlay/91132a78500e0ffae0b7d1ab22e9e44f700fe4bda2e6d73f2168d649abc88f10/merged major:0 minor:284 fsType:overlay blockSize:0} overlay_0-289:{mountpoint:/var/lib/containers/storage/overlay/f97c5ac864d991bb3a081e3ca8cfb5ec260bc79b54b4b42e1ae38b7f41043059/merged major:0 minor:289 fsType:overlay blockSize:0} overlay_0-294:{mountpoint:/var/lib/containers/storage/overlay/c372303bae8b8f6ce2bf5d6bc298a725482800c65d3bc4c8c2d7a3976b9bc682/merged major:0 minor:294 fsType:overlay blockSize:0} overlay_0-299:{mountpoint:/var/lib/containers/storage/overlay/db01a401d6ff1f71df3885333a4aec07083e5905b58369f9e8073695d78407a0/merged major:0 minor:299 fsType:overlay blockSize:0} overlay_0-305:{mountpoint:/var/lib/containers/storage/overlay/68f2d06b818c389d4c21eafa5784864ee48be784e648de9d968edeb07aa14a7b/merged major:0 minor:305 fsType:overlay blockSize:0} overlay_0-322:{mountpoint:/var/lib/containers/storage/overlay/b4918dad30bd5fa88ed0e71d4f9853a35062e964674f6a55e1ae83f66ed8bed4/merged major:0 minor:322 fsType:overlay blockSize:0} overlay_0-328:{mountpoint:/var/lib/containers/storage/overlay/22fbe305c474eab718e91f3f0708ad98b8de7d6e4f10c23e83e7135d62e1ec21/merged major:0 minor:328 fsType:overlay blockSize:0} overlay_0-330:{mountpoint:/var/lib/containers/storage/overlay/ebda0b04513e3ddc0e0a7efc4fbd5fd871b33db4a1c417a092317d0f7765abb0/merged major:0 minor:330 fsType:overlay blockSize:0} overlay_0-332:{mountpoint:/var/lib/containers/storage/overlay/70ffeb3ffc1d0d87a9858dd98ebf3bc18ef91d544f9432abb0a7e0dacb1fac21/merged major:0 minor:332 fsType:overlay blockSize:0} overlay_0-338:{mountpoint:/var/lib/containers/storage/overlay/cd23d8f892af0873f00715cbd5c4595389e9442cfbcdf5019532229f55ef669d/merged major:0 minor:338 fsType:overlay blockSize:0} overlay_0-343:{mountpoint:/var/lib/containers/storage/overlay/a72220a4bd38cd6a61ee81459ff586f10d7d9d2252bf64c032441791b215fef7/merged major:0 minor:343 fsType:overlay blockSize:0} overlay_0-348:{mountpoint:/var/lib/containers/storage/overlay/c3c8cc764282e4ef0c1903ea530c69bc237992fd4872e2047450e5c8b1e064a9/merged major:0 minor:348 fsType:overlay blockSize:0} overlay_0-353:{mountpoint:/var/lib/containers/storage/overlay/11143ff19c43df2040140496a366c8759a34633a239bfbcf8a02757b31a3e042/merged major:0 minor:353 fsType:overlay blockSize:0} overlay_0-358:{mountpoint:/var/lib/containers/storage/overlay/a77b32fd6c593e483243eef927ff69903a7a3654a32fe55baa9451e98bd38416/merged major:0 minor:358 fsType:overlay blockSize:0} overlay_0-381:{mountpoint:/var/lib/containers/storage/overlay/fee18b4ef83ef136dac9c1551471bfe8b214dfb2337d3dc2bfca49a961492266/merged major:0 minor:381 fsType:overlay blockSize:0} overlay_0-383:{mountpoint:/var/lib/containers/storage/overlay/b9e2dc3e6d4733b8010c33debf3915ed45e37112055799194023870f4a4d1dc9/merged major:0 minor:383 fsType:overlay blockSize:0} overlay_0-385:{mountpoint:/var/lib/containers/storage/overlay/691370bd2bcbade83da1c546e52abee00e79a2d705f8947e0c124fefaa96ca84/merged major:0 minor:385 fsType:overlay blockSize:0} overlay_0-391:{mountpoint:/var/lib/containers/storage/overlay/796a2fbdab25570dc44ff2d5083b26815651470912646f0294c412b94f7612cb/merged major:0 minor:391 fsType:overlay blockSize:0} overlay_0-396:{mountpoint:/var/lib/containers/storage/overlay/8024b19861e58b4450700f3cd0764381c1521b0405c2a2f31ace7c690fea61a5/merged major:0 minor:396 fsType:overlay blockSize:0} overlay_0-401:{mountpoint:/var/lib/containers/storage/overlay/82b9a7bf464bfba677b962d51f394006ed90996dc0cc34988efe4d5c73cb58a0/merged major:0 minor:401 fsType:overlay blockSize:0} overlay_0-406:{mountpoint:/var/lib/containers/storage/overlay/fc25e50e67ac0a802af054ec7ae22fe633efd9df06aef1de0dab197482276716/merged major:0 minor:406 fsType:overlay blockSize:0} overlay_0-423:{mountpoint:/var/lib/containers/storage/overlay/02a769e855a5f1606b577fd021a9716c0ffb48e639fa95dc3bd928742748b091/merged major:0 minor:423 fsType:overlay blockSize:0} overlay_0-425:{mountpoint:/var/lib/containers/storage/overlay/9c2c00151cc27e9561846d59f50d59f5c7f6c04a512f30fc1e170979804682b2/merged major:0 minor:425 fsType:overlay blockSize:0} overlay_0-427:{mountpoint:/var/lib/containers/storage/overlay/08c07675e7c49057ae06f80da9da306e2332e1f21cc32f2c212175191d7537b4/merged major:0 minor:427 fsType:overlay blockSize:0} overlay_0-429:{mountpoint:/var/lib/containers/storage/overlay/4a0ec118a0458941b42a36fdc46d798a3d72cd15570578ad49deab1958b9e2e7/merged major:0 minor:429 fsType:overlay blockSize:0} overlay_0-431:{mountpoint:/var/lib/containers/storage/overlay/a4aa9951d52891114a925222cc3c9da1f330e2d1eb217b81442bb7a2c1b2c5a6/merged major:0 minor:431 fsType:overlay blockSize:0} overlay_0-437:{mountpoint:/var/lib/containers/storage/overlay/f13cdb9e13b5a76359afc44b11d1598609e174efe0d958e94b905ebc0e078a81/merged major:0 minor:437 fsType:overlay blockSize:0} overlay_0-442:{mountpoint:/var/lib/containers/storage/overlay/c7a1be87fd1179c35eaed203e2423fccda7223b8bdc7e439943206e6fba38bc6/merged major:0 minor:442 fsType:overlay blockSize:0} overlay_0-50:{mountpoint:/var/lib/containers/storage/overlay/81320cf799f0b9ab415db2562c7937b131e9b081f5b2d62f98616661071e63a8/merged major:0 minor:50 fsType:overlay blockSize:0} overlay_0-57:{mountpoint:/var/lib/containers/storage/overlay/c5ca15c7c59df4639768ca236eee0cdf5187db1646bbfb05eccf9945f873f941/merged major:0 minor:57 fsType:overlay blockSize:0} overlay_0-70:{mountpoint:/var/lib/containers/storage/overlay/ca8c593f99d39d1c014ad13ad9a5baad979471325760f2861ef711b777d904b1/merged major:0 minor:70 fsType:overlay blockSize:0} overlay_0-74:{mountpoint:/var/lib/containers/storage/overlay/d94e43f6f9591fb558b72781f58257ad13dd7da0569e7d395e3ad3b72e7bf61d/merged major:0 minor:74 fsType:overlay blockSize:0} overlay_0-78:{mountpoint:/var/lib/containers/storage/overlay/ad6699120b734f6601624b79974b647d3fd46a0a89fc09b9f692455ef0b16597/merged major:0 minor:78 fsType:overlay blockSize:0} overlay_0-82:{mountpoint:/var/lib/containers/storage/overlay/19af6d6c623a8ede6b4001e716d83c0c66f979d3b1d30bdfb5ab0b5a647391b0/merged major:0 minor:82 fsType:overlay blockSize:0} overlay_0-84:{mountpoint:/var/lib/containers/storage/overlay/74caa0faf40fcdfe4a0f074de22be36e6012fe3fd44737e3d4425a8008d40071/merged major:0 minor:84 fsType:overlay blockSize:0} overlay_0-88:{mountpoint:/var/lib/containers/storage/overlay/c1129c402cb2b34de8234e539d62b2cc3e565af4f9c69077659c494e8345b333/merged major:0 minor:88 fsType:overlay blockSize:0} overlay_0-92:{mountpoint:/var/lib/containers/storage/overlay/f08e2e7b18b8c3acbcd659381af76013f082eb44f90c7c349999bda7cbfb930f/merged major:0 minor:92 fsType:overlay blockSize:0} overlay_0-96:{mountpoint:/var/lib/containers/storage/overlay/0c271ac69a64ce24d48a24384a337f28c7c6e603c6d092a725d45725de4103a5/merged major:0 minor:96 fsType:overlay blockSize:0}]
Oct 30 21:54:53.302787 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.302165    6133 manager.go:217] Machine: {Timestamp:2026-10-30 21:54:53.301676145 +0000 UTC m=+0.062432026 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3115358 MemoryCapacity:33164337152 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2cd0a68206bbc02ac459ae78208eb8 SystemUUID:ec2cd0a6-8206-bbc0-2ac4-59ae78208eb8 BootID:aa5b2c7a-2c32-463a-a8cc-603cc9c8ac89 Filesystems:[{Device:overlay_0-119 DeviceMajor:0 DeviceMinor:119 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-124 DeviceMajor:0 DeviceMinor:124 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/7ddfa67d-20a0-473d-937f-25fd8e0ba80e/volumes/kubernetes.io~secret/node-exporter-kube-rbac-proxy-config DeviceMajor:0 DeviceMinor:242 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-383 DeviceMajor:0 DeviceMinor:383 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/532658b6-c86e-4feb-89cb-14a91316a7a0/volumes/kubernetes.io~secret/metrics-certs DeviceMajor:0 DeviceMinor:415 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-78 DeviceMajor:0 DeviceMinor:78 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-177 DeviceMajor:0 DeviceMinor:177 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-182 DeviceMajor:0 DeviceMinor:182 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/c0fd54d2f9eade40d52d73aa5fd6dfb74c9a1ed40c855e3e0773cb680ead5d10/userdata/shm DeviceMajor:0 DeviceMinor:207 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-343 DeviceMajor:0 DeviceMinor:343 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/bd93518f-2f96-4483-9e92-513109dd37cb/volumes/kubernetes.io~projected/kube-api-access-qrt69 DeviceMajor:0 DeviceMinor:378 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/var/lib/kubelet/pods/bebdb584-1a96-453d-8a6d-b332e47bec3d/volumes/kubernetes.io~empty-dir/etc-tuned DeviceMajor:0 DeviceMinor:54 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/9ce524660d905f015a7f4f67394797c49fed963f767dec7a56702ccf03654c8b/userdata/shm DeviceMajor:0 DeviceMinor:76 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/539e430a94bd9fa3fb11634f988f610e9eced549a966b67a5185fe5759aad92c/userdata/shm DeviceMajor:0 DeviceMinor:98 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/fbd8f09b-9962-4910-8b25-267cbbb3ea5d/volumes/kubernetes.io~secret/secret-thanos-querier-tls DeviceMajor:0 DeviceMinor:315 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/bd93518f-2f96-4483-9e92-513109dd37cb/volumes/kubernetes.io~secret/secret-metrics-client-certs DeviceMajor:0 DeviceMinor:373 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/bd93518f-2f96-4483-9e92-513109dd37cb/volume-subpaths/web-config/init-config-reloader/2 DeviceMajor:0 DeviceMinor:374 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/446beacf-70a0-4d47-96ea-bbe1ca4a5f28/volumes/kubernetes.io~projected/kube-api-access-mkrsw DeviceMajor:0 DeviceMinor:66 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-82 DeviceMajor:0 DeviceMinor:82 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/17da628d-31f5-456c-ad37-52c884d22cf4/volumes/kubernetes.io~projected/kube-api-access-bw6lg DeviceMajor:0 DeviceMinor:248 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-258 DeviceMajor:0 DeviceMinor:258 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/bd93518f-2f96-4483-9e92-513109dd37cb/volumes/kubernetes.io~secret/secret-prometheus-k8s-tls DeviceMajor:0 DeviceMinor:368 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-396 DeviceMajor:0 DeviceMinor:396 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:32 Capacity:16582168576 Type:vfs Inodes:1048576 HasInodes:true} {Device:/var/lib/kubelet/pods/fe472529-1294-46a1-a666-dc00d5d37eca/volumes/kubernetes.io~projected/kube-api-access-nbqv6 DeviceMajor:0 DeviceMinor:65 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/efaaacbcd02ded84d312f70628a2cf5b042af9118274819e3ab06919565b3f46/userdata/shm DeviceMajor:0 DeviceMinor:252 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-284 DeviceMajor:0 DeviceMinor:284 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/bd93518f-2f96-4483-9e92-513109dd37cb/volumes/kubernetes.io~secret/config DeviceMajor:0 DeviceMinor:363 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-74 DeviceMajor:0 DeviceMinor:74 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-164 DeviceMajor:0 DeviceMinor:164 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-358 DeviceMajor:0 DeviceMinor:358 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/bd93518f-2f96-4483-9e92-513109dd37cb/volumes/kubernetes.io~secret/thanos-prometheus-http-client-file DeviceMajor:0 DeviceMinor:372 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-192 DeviceMajor:0 DeviceMinor:192 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-211 DeviceMajor:0 DeviceMinor:211 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-289 DeviceMajor:0 DeviceMinor:289 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-381 DeviceMajor:0 DeviceMinor:381 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/bd93518f-2f96-4483-9e92-513109dd37cb/volume-subpaths/web-config/config-reloader/2 DeviceMajor:0 DeviceMinor:374 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-391 DeviceMajor:0 DeviceMinor:391 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-162 DeviceMajor:0 DeviceMinor:162 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/484e37a0-fd1d-4fd0-a200-939b8945984b/volumes/kubernetes.io~secret/openshift-state-metrics-kube-rbac-proxy-config DeviceMajor:0 DeviceMinor:236 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/bd93518f-2f96-4483-9e92-513109dd37cb/volume-subpaths/web-config/prometheus/15 DeviceMajor:0 DeviceMinor:374 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/81edfd255828d18cc1f4fca31e927255f649b6b1bcbba23bb65a1819dd9a9b65/userdata/shm DeviceMajor:0 DeviceMinor:246 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/fbd8f09b-9962-4910-8b25-267cbbb3ea5d/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy-rules DeviceMajor:0 DeviceMinor:318 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-423 DeviceMajor:0 DeviceMinor:423 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-114 DeviceMajor:0 DeviceMinor:114 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-230 DeviceMajor:0 DeviceMinor:230 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/83f5c0d6735a2e35b4e005074f2c7f583f92ea7a94d680c9c430b8d1df568d4c/userdata/shm DeviceMajor:0 DeviceMinor:379 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/6192d2be-bba7-4099-b54c-e7e9b8d75389/volumes/kubernetes.io~projected/kube-api-access-jw4wk DeviceMajor:0 DeviceMinor:201 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-348 DeviceMajor:0 DeviceMinor:348 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/5d5057fc-dd8a-4dce-b8bb-70b2dc3b074c/volumes/kubernetes.io~projected/kube-api-access-kncjh DeviceMajor:0 DeviceMinor:62 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/bebdb584-1a96-453d-8a6d-b332e47bec3d/volumes/kubernetes.io~projected/kube-api-access-r4qz4 DeviceMajor:0 DeviceMinor:63 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-137 DeviceMajor:0 DeviceMinor:137 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/e87446335be5893b70428a03a70ad83dac422290b1eff305dbe509ee7e35170d/userdata/shm DeviceMajor:0 DeviceMinor:262 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-282 DeviceMajor:0 DeviceMinor:282 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/fbd8f09b-9962-4910-8b25-267cbbb3ea5d/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy-web DeviceMajor:0 DeviceMinor:310 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/fbd8f09b-9962-4910-8b25-267cbbb3ea5d/volumes/kubernetes.io~secret/secret-grpc-tls DeviceMajor:0 DeviceMinor:314 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/fbd8f09b-9962-4910-8b25-267cbbb3ea5d/volumes/kubernetes.io~projected/kube-api-access-zzw6w DeviceMajor:0 DeviceMinor:319 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-84 DeviceMajor:0 DeviceMinor:84 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/8ce50405433f59d6286d2bbf2f1008372c665afdfb13349ef7546cf0b1be08a4/userdata/shm DeviceMajor:0 DeviceMinor:205 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/9d2cced5-9ceb-421f-b831-8370a5ee0ca9/volumes/kubernetes.io~projected/kube-api-access-w78hr DeviceMajor:0 DeviceMinor:325 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/bd93518f-2f96-4483-9e92-513109dd37cb/volumes/kubernetes.io~secret/secret-prometheus-k8s-kube-rbac-proxy-web DeviceMajor:0 DeviceMinor:369 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/bd93518f-2f96-4483-9e92-513109dd37cb/volumes/kubernetes.io~secret/secret-kube-rbac-proxy DeviceMajor:0 DeviceMinor:370 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-385 DeviceMajor:0 DeviceMinor:385 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-157 DeviceMajor:0 DeviceMinor:157 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-328 DeviceMajor:0 DeviceMinor:328 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-406 DeviceMajor:0 DeviceMinor:406 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-88 DeviceMajor:0 DeviceMinor:88 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-139 DeviceMajor:0 DeviceMinor:139 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/d65ef881919ac2a78210156713c755f348c9da7cd09bd6af266bbdfa0cb76498/userdata/shm DeviceMajor:0 DeviceMinor:326 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/446beacf-70a0-4d47-96ea-bbe1ca4a5f28/volumes/kubernetes.io~secret/ovn-node-metrics-cert DeviceMajor:0 DeviceMinor:55 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/fc4a67d9-8c7c-4f71-9a8a-2bba09329bc6/volumes/kubernetes.io~projected/kube-api-access-jp25c DeviceMajor:0 DeviceMinor:67 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/210e318062886f16a6ccfb46ef82db381f07dd4d7d0518075c707f0853ed9ea9/userdata/shm DeviceMajor:0 DeviceMinor:90 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-169 DeviceMajor:0 DeviceMinor:169 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/6192d2be-bba7-4099-b54c-e7e9b8d75389/volumes/kubernetes.io~secret/insights-runtime-extractor-tls DeviceMajor:0 DeviceMinor:45 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-217 DeviceMajor:0 DeviceMinor:217 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/7ddfa67d-20a0-473d-937f-25fd8e0ba80e/volumes/kubernetes.io~projected/kube-api-access-wsnr5 DeviceMajor:0 DeviceMinor:249 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-299 DeviceMajor:0 DeviceMinor:299 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/613f514c-6011-4936-8df5-a59b960cd3a6/volumes/kubernetes.io~projected/kube-api-access-h6b6r DeviceMajor:0 DeviceMinor:60 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/ea0896e86f6df5158e1a1ecc5b41aa8e849df46707c6cc80f76940fc2dadcd09/userdata/shm DeviceMajor:0 DeviceMinor:209 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-219 DeviceMajor:0 DeviceMinor:219 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-225 DeviceMajor:0 DeviceMinor:225 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-442 DeviceMajor:0 DeviceMinor:442 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/3cf409dd46f74bbfa1745012a0345fc7230a78cf391809e4d473e6cf35376e9e/userdata/shm DeviceMajor:0 DeviceMinor:94 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/7ddfa67d-20a0-473d-937f-25fd8e0ba80e/volumes/kubernetes.io~secret/node-exporter-tls DeviceMajor:0 DeviceMinor:243 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-332 DeviceMajor:0 DeviceMinor:332 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:24 Capacity:16582168576 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/4621048c-230e-441f-85fc-423e4ba525ec/volumes/kubernetes.io~projected/kube-api-access-9f2kg DeviceMajor:0 DeviceMinor:204 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-271 DeviceMajor:0 DeviceMinor:271 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/0bc4cf08b77946fc12c13d79e800268cebb120d1de733fa971a67b52e17c9336/userdata/shm DeviceMajor:0 DeviceMinor:421 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-437 DeviceMajor:0 DeviceMinor:437 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-50 DeviceMajor:0 DeviceMinor:50 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/2bcee03a-9b40-4155-8d81-43dece439162/volumes/kubernetes.io~secret/agent-certs DeviceMajor:0 DeviceMinor:53 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/fbd8f09b-9962-4910-8b25-267cbbb3ea5d/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy DeviceMajor:0 DeviceMinor:317 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/a1a23f1cc1936fcf6f0fbc3af4009a0cd4a17790ed28b13c1dfc5accffe8f597/userdata/shm DeviceMajor:0 DeviceMinor:47 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/bebdb584-1a96-453d-8a6d-b332e47bec3d/volumes/kubernetes.io~empty-dir/tmp DeviceMajor:0 DeviceMinor:56 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-57 DeviceMajor:0 DeviceMinor:57 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-143 DeviceMajor:0 DeviceMinor:143 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/fbd8f09b-9962-4910-8b25-267cbbb3ea5d/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy-metrics DeviceMajor:0 DeviceMinor:316 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/9d2cced5-9ceb-421f-b831-8370a5ee0ca9/volumes/kubernetes.io~secret/monitoring-plugin-cert DeviceMajor:0 DeviceMinor:324 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/bd93518f-2f96-4483-9e92-513109dd37cb/volumes/kubernetes.io~secret/secret-prometheus-k8s-thanos-sidecar-tls DeviceMajor:0 DeviceMinor:371 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/bd93518f-2f96-4483-9e92-513109dd37cb/volumes/kubernetes.io~projected/tls-assets DeviceMajor:0 DeviceMinor:375 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/532658b6-c86e-4feb-89cb-14a91316a7a0/volumes/kubernetes.io~projected/kube-api-access-r6wtm DeviceMajor:0 DeviceMinor:59 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/032e17cbfd6990eb3c9bc358001153b8ae4e45b54c916d214281a29bb48fe869/userdata/shm DeviceMajor:0 DeviceMinor:68 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-146 DeviceMajor:0 DeviceMinor:146 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-213 DeviceMajor:0 DeviceMinor:213 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/af83b188-09cb-4621-a9da-be33856d19f0/volumes/kubernetes.io~secret/original-pull-secret DeviceMajor:0 DeviceMinor:411 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-425 DeviceMajor:0 DeviceMinor:425 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/e963d0a97dfea14806021bbebf9320c7147bdee479c686f3a196b1ca0ed1b577/userdata/shm DeviceMajor:0 DeviceMinor:80 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-132 DeviceMajor:0 DeviceMinor:132 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-215 DeviceMajor:0 DeviceMinor:215 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/484e37a0-fd1d-4fd0-a200-939b8945984b/volumes/kubernetes.io~secret/openshift-state-metrics-tls DeviceMajor:0 DeviceMinor:240 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/484e37a0-fd1d-4fd0-a200-939b8945984b/volumes/kubernetes.io~projected/kube-api-access-n87x8 DeviceMajor:0 DeviceMinor:241 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-260 DeviceMajor:0 DeviceMinor:260 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-34 DeviceMajor:0 DeviceMinor:34 Capacity:6422528 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:overlay_0-100 DeviceMajor:0 DeviceMinor:100 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-127 DeviceMajor:0 DeviceMinor:127 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/209b4e58-e6c2-48f1-8589-f7b6be79720b/volumes/kubernetes.io~secret/metrics-tls DeviceMajor:0 DeviceMinor:197 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-330 DeviceMajor:0 DeviceMinor:330 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-92 DeviceMajor:0 DeviceMinor:92 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-96 DeviceMajor:0 DeviceMinor:96 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-102 DeviceMajor:0 DeviceMinor:102 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-269 DeviceMajor:0 DeviceMinor:269 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-305 DeviceMajor:0 DeviceMinor:305 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-431 DeviceMajor:0 DeviceMinor:431 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/a3f130547617214efc2a1c06555379109c660bad2501f887be595e3d279ea8a4/userdata/shm DeviceMajor:0 DeviceMinor:49 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/ee565a8a0022cec61d84d7110490393c151780d6fd4c9eb513e311b6a19048d8/userdata/shm DeviceMajor:0 DeviceMinor:86 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-264 DeviceMajor:0 DeviceMinor:264 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-294 DeviceMajor:0 DeviceMinor:294 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/bd93518f-2f96-4483-9e92-513109dd37cb/volumes/kubernetes.io~secret/web-config DeviceMajor:0 DeviceMinor:374 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/4397165a20a420aac8ad6d0f822e75d8972b23e73db86d9ba549c829116c4480/userdata/shm DeviceMajor:0 DeviceMinor:419 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-427 DeviceMajor:0 DeviceMinor:427 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-429 DeviceMajor:0 DeviceMinor:429 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-277 DeviceMajor:0 DeviceMinor:277 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-338 DeviceMajor:0 DeviceMinor:338 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-70 DeviceMajor:0 DeviceMinor:70 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/4efea9f6053c04c56805d6877572edc5c4aa453e6c534f35eefcdfdf5cc9947b/userdata/shm DeviceMajor:0 DeviceMinor:72 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/15e1de31-92a1-4179-8ec1-400346880d1d/volumes/kubernetes.io~projected/kube-api-access-lgk4w DeviceMajor:0 DeviceMinor:64 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/17da628d-31f5-456c-ad37-52c884d22cf4/volumes/kubernetes.io~secret/kube-state-metrics-kube-rbac-proxy-config DeviceMajor:0 DeviceMinor:244 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/bfb89131d151d9c07894bf934194c2ebb93199fbe0f6297f748d4cdf54d68975/userdata/shm DeviceMajor:0 DeviceMinor:320 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-401 DeviceMajor:0 DeviceMinor:401 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/9bd7faef61deec19be60215e6651eb7da94b352c4e82286d682c30c02b91f953/userdata/shm DeviceMajor:0 DeviceMinor:417 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-187 DeviceMajor:0 DeviceMinor:187 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/17da628d-31f5-456c-ad37-52c884d22cf4/volumes/kubernetes.io~secret/kube-state-metrics-tls DeviceMajor:0 DeviceMinor:245 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-322 DeviceMajor:0 DeviceMinor:322 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-353 DeviceMajor:0 DeviceMinor:353 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/446beacf-70a0-4d47-96ea-bbe1ca4a5f28/volume-subpaths/run-systemd/ovnkube-controller/6 DeviceMajor:0 DeviceMinor:26 Capacity:6632869888 Type:vfs Inodes:819200 HasInodes:true} {Device:/var/lib/kubelet/pods/209b4e58-e6c2-48f1-8589-f7b6be79720b/volumes/kubernetes.io~projected/kube-api-access-mrtmr DeviceMajor:0 DeviceMinor:202 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-250 DeviceMajor:0 DeviceMinor:250 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/bd93518f-2f96-4483-9e92-513109dd37cb/volumes/kubernetes.io~empty-dir/config-out DeviceMajor:0 DeviceMinor:376 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/bd93518f-2f96-4483-9e92-513109dd37cb/volumes/kubernetes.io~secret/secret-grpc-tls DeviceMajor:0 DeviceMinor:377 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:26 Capacity:6632869888 Type:vfs Inodes:819200 HasInodes:true} {Device:/var/lib/kubelet/pods/3e60fceb-8682-45cc-a472-f3d7017c3395/volumes/kubernetes.io~projected/kube-api-access-6bsvp DeviceMajor:0 DeviceMinor:61 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-109 DeviceMajor:0 DeviceMinor:109 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/4621048c-230e-441f-85fc-423e4ba525ec/volumes/kubernetes.io~secret/cert DeviceMajor:0 DeviceMinor:203 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/32ae0d42-4b76-47b7-b9d6-26b09f00ba5d/volumes/kubernetes.io~projected/kube-api-access-ljsqq DeviceMajor:0 DeviceMinor:416 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-104 DeviceMajor:0 DeviceMinor:104 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:0bc4cf08b77946f MacAddress:1a:02:56:62:63:9a Speed:10000 Mtu:8901} {Name:4397165a20a420a MacAddress:f2:34:73:30:ee:65 Speed:10000 Mtu:8901} {Name:81edfd255828d18 MacAddress:5e:4d:8d:f1:64:c4 Speed:10000 Mtu:8901} {Name:83f5c0d6735a2e3 MacAddress:86:0c:02:38:29:bb Speed:10000 Mtu:8901} {Name:8ce50405433f59d MacAddress:92:61:47:10:ed:e8 Speed:10000 Mtu:8901} {Name:9bd7faef61deec1 MacAddress:a2:97:55:59:81:50 Speed:10000 Mtu:8901} {Name:bfb89131d151d9c MacAddress:6a:74:46:3f:a1:d3 Speed:10000 Mtu:8901} {Name:br-ex MacAddress:02:ff:ca:14:98:c9 Speed:0 Mtu:9001} {Name:br-int MacAddress:16:90:58:99:fa:82 Speed:0 Mtu:8901} {Name:c0fd54d2f9eade4 MacAddress:4a:7a:d8:52:e2:97 Speed:10000 Mtu:8901} {Name:d65ef881919ac2a MacAddress:6a:fa:0e:db:ab:56 Speed:10000 Mtu:8901} {Name:e87446335be5893 MacAddress:5a:ad:94:ae:4f:1b Speed:10000 Mtu:8901} {Name:ea0896e86f6df51 MacAddress:46:61:76:85:77:07 Speed:10000 Mtu:8901} {Name:ens5 MacAddress:02:ff:ca:14:98:c9 Speed:0 Mtu:9001} {Name:genev_sys_6081 MacAddress:ca:e3:b8:a6:05:a3 Speed:0 Mtu:65000} {Name:ovn-k8s-mp0 MacAddress:0a:58:0a:86:00:02 Speed:0 Mtu:8901} {Name:ovs-system MacAddress:5a:bd:df:2b:0b:ea Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164337152 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Oct 30 21:54:53.302787 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.302616    6133 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Oct 30 21:54:53.302787 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.302666    6133 manager.go:233] Version: {KernelVersion:5.14.0-687.60.1.el9_8.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.8.20261023-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Oct 30 21:54:53.303092 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.303059    6133 container_manager_linux.go:275] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Oct 30 21:54:53.303246 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.303096    6133 container_manager_linux.go:280] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-143-246.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"/system.slice","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{},"system-reserved-compressible":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Oct 30 21:54:53.303287 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.303249    6133 topology_manager.go:143] "Creating topology manager with none policy"
Oct 30 21:54:53.303287 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.303257    6133 container_manager_linux.go:311] "Creating device plugin manager"
Oct 30 21:54:53.303287 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.303269    6133 manager.go:144] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Oct 30 21:54:53.303366 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.303289    6133 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Oct 30 21:54:53.303407 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.303399    6133 container_manager_linux.go:320] "Creating Dynamic Resource Allocation (DRA) manager"
Oct 30 21:54:53.303585 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.303575    6133 state_mem.go:41] "Initialized" logger="CPUManager state memory"
Oct 30 21:54:53.303697 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.303689    6133 server.go:1235] "Using root directory" path="/var/lib/kubelet"
Oct 30 21:54:53.303729 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.303725    6133 kubelet.go:493] "Attempting to sync node with API server"
Oct 30 21:54:53.303760 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.303739    6133 kubelet.go:394] "Adding static pod path" path="/etc/kubernetes/manifests"
Oct 30 21:54:53.303760 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.303750    6133 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Oct 30 21:54:53.303760 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.303760    6133 kubelet.go:405] "Adding apiserver pod source"
Oct 30 21:54:53.303880 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.303769    6133 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Oct 30 21:54:53.304532 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.304516    6133 state_checkpoint.go:81] "State checkpoint: restored pod resource state from checkpoint"
Oct 30 21:54:53.304532 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.304528    6133 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Oct 30 21:54:53.305106 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.305086    6133 kuberuntime_manager.go:304] "Container runtime initialized" containerRuntime="cri-o" version="1.35.10-3.rhaos4.22.git9383502.el9" apiVersion="v1"
Oct 30 21:54:53.308063 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.308048    6133 state_checkpoint.go:81] "State checkpoint: restored pod resource state from checkpoint"
Oct 30 21:54:53.308063 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.308062    6133 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Oct 30 21:54:53.308172 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.308162    6133 certificate_store.go:147] "Loading cert/key pair from a file" filePath="/var/lib/kubelet/pki/kubelet-server-current.pem"
Oct 30 21:54:53.308364 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.308353    6133 kubelet.go:963] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Oct 30 21:54:53.308393 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.308382    6133 kubelet.go:990] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Oct 30 21:54:53.308485 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.308476    6133 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Oct 30 21:54:53.308515 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.308491    6133 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Oct 30 21:54:53.308515 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.308498    6133 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Oct 30 21:54:53.308515 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.308504    6133 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Oct 30 21:54:53.308515 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.308509    6133 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Oct 30 21:54:53.308515 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.308515    6133 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Oct 30 21:54:53.308658 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.308520    6133 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Oct 30 21:54:53.308658 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.308527    6133 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Oct 30 21:54:53.308658 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.308535    6133 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Oct 30 21:54:53.308658 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.308541    6133 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Oct 30 21:54:53.308658 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.308556    6133 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Oct 30 21:54:53.308658 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.308574    6133 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Oct 30 21:54:53.308658 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.308604    6133 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Oct 30 21:54:53.308658 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.308610    6133 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Oct 30 21:54:53.312017 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.312003    6133 server.go:1263] "Started kubelet"
Oct 30 21:54:53.312364 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.312102    6133 server.go:182] "Starting to listen" address="0.0.0.0" port=10250
Oct 30 21:54:53.313697 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.312154    6133 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Oct 30 21:54:53.313697 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.313145    6133 server_v1.go:49] "podresources" method="list" useActivePods=true
Oct 30 21:54:53.313287 ip-10-0-143-246 systemd[1]: Started Kubernetes Kubelet.
Oct 30 21:54:53.314116 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.313731    6133 server.go:254] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Oct 30 21:54:53.315757 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.315473    6133 server.go:317] "Adding debug handlers to kubelet server"
Oct 30 21:54:53.319495 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.318367    6133 fs_resource_analyzer.go:69] "Starting FS ResourceAnalyzer"
Oct 30 21:54:53.319495 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.318440    6133 volume_manager.go:309] "The desired_state_of_world populator starts"
Oct 30 21:54:53.319495 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.318453    6133 volume_manager.go:311] "Starting Kubelet Volume Manager"
Oct 30 21:54:53.319495 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.318516    6133 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Oct 30 21:54:53.319495 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.318536    6133 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Oct 30 21:54:53.319495 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.318558    6133 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-10-29 21:48:44 +0000 UTC" deadline="2028-03-29 05:13:20.377632163 +0000 UTC"
Oct 30 21:54:53.319495 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.318578    6133 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12367h18m27.059056962s"
Oct 30 21:54:53.319495 ip-10-0-143-246 kubenswrapper[6133]: E1030 21:54:53.318646    6133 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-143-246.ec2.internal\" not found"
Oct 30 21:54:53.321871 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.320312    6133 factory.go:55] Registering systemd factory
Oct 30 21:54:53.321871 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.320333    6133 factory.go:223] Registration of the systemd container factory successfully
Oct 30 21:54:53.321871 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.320598    6133 factory.go:153] Registering CRI-O factory
Oct 30 21:54:53.321871 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.320614    6133 factory.go:223] Registration of the crio container factory successfully
Oct 30 21:54:53.321871 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.320681    6133 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Oct 30 21:54:53.321871 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.320706    6133 factory.go:103] Registering Raw factory
Oct 30 21:54:53.321871 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.320721    6133 manager.go:1199] Started watching for new ooms in manager
Oct 30 21:54:53.321871 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.321114    6133 manager.go:319] Starting recovery of all containers
Oct 30 21:54:53.321871 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.321408    6133 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7ddfa67d-20a0-473d-937f-25fd8e0ba80e" volumeName="kubernetes.io/configmap/7ddfa67d-20a0-473d-937f-25fd8e0ba80e-metrics-client-ca" seLinuxMountContext=""
Oct 30 21:54:53.321871 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.321462    6133 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="bd93518f-2f96-4483-9e92-513109dd37cb" volumeName="kubernetes.io/empty-dir/bd93518f-2f96-4483-9e92-513109dd37cb-config-out" seLinuxMountContext=""
Oct 30 21:54:53.321871 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.321475    6133 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="fe472529-1294-46a1-a666-dc00d5d37eca" volumeName="kubernetes.io/configmap/fe472529-1294-46a1-a666-dc00d5d37eca-serviceca" seLinuxMountContext=""
Oct 30 21:54:53.321871 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.321488    6133 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="17da628d-31f5-456c-ad37-52c884d22cf4" volumeName="kubernetes.io/empty-dir/17da628d-31f5-456c-ad37-52c884d22cf4-volume-directive-shadow" seLinuxMountContext=""
Oct 30 21:54:53.321871 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.321500    6133 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="17da628d-31f5-456c-ad37-52c884d22cf4" volumeName="kubernetes.io/secret/17da628d-31f5-456c-ad37-52c884d22cf4-kube-state-metrics-tls" seLinuxMountContext=""
Oct 30 21:54:53.321871 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.321514    6133 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="2bcee03a-9b40-4155-8d81-43dece439162" volumeName="kubernetes.io/secret/2bcee03a-9b40-4155-8d81-43dece439162-agent-certs" seLinuxMountContext=""
Oct 30 21:54:53.321871 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.321526    6133 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="bd93518f-2f96-4483-9e92-513109dd37cb" volumeName="kubernetes.io/secret/bd93518f-2f96-4483-9e92-513109dd37cb-secret-prometheus-k8s-thanos-sidecar-tls" seLinuxMountContext=""
Oct 30 21:54:53.321871 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.321538    6133 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="fbd8f09b-9962-4910-8b25-267cbbb3ea5d" volumeName="kubernetes.io/projected/fbd8f09b-9962-4910-8b25-267cbbb3ea5d-kube-api-access-zzw6w" seLinuxMountContext=""
Oct 30 21:54:53.321871 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.321553    6133 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="17da628d-31f5-456c-ad37-52c884d22cf4" volumeName="kubernetes.io/configmap/17da628d-31f5-456c-ad37-52c884d22cf4-kube-state-metrics-custom-resource-state-configmap" seLinuxMountContext=""
Oct 30 21:54:53.321871 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.321564    6133 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="17da628d-31f5-456c-ad37-52c884d22cf4" volumeName="kubernetes.io/projected/17da628d-31f5-456c-ad37-52c884d22cf4-kube-api-access-bw6lg" seLinuxMountContext=""
Oct 30 21:54:53.321871 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.321576    6133 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="532658b6-c86e-4feb-89cb-14a91316a7a0" volumeName="kubernetes.io/secret/532658b6-c86e-4feb-89cb-14a91316a7a0-metrics-certs" seLinuxMountContext=""
Oct 30 21:54:53.321871 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.321588    6133 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7ddfa67d-20a0-473d-937f-25fd8e0ba80e" volumeName="kubernetes.io/secret/7ddfa67d-20a0-473d-937f-25fd8e0ba80e-node-exporter-kube-rbac-proxy-config" seLinuxMountContext=""
Oct 30 21:54:53.321871 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.321600    6133 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="bd93518f-2f96-4483-9e92-513109dd37cb" volumeName="kubernetes.io/configmap/bd93518f-2f96-4483-9e92-513109dd37cb-configmap-serving-certs-ca-bundle" seLinuxMountContext=""
Oct 30 21:54:53.321871 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.321669    6133 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="bd93518f-2f96-4483-9e92-513109dd37cb" volumeName="kubernetes.io/secret/bd93518f-2f96-4483-9e92-513109dd37cb-secret-grpc-tls" seLinuxMountContext=""
Oct 30 21:54:53.321871 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.321699    6133 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="446beacf-70a0-4d47-96ea-bbe1ca4a5f28" volumeName="kubernetes.io/configmap/446beacf-70a0-4d47-96ea-bbe1ca4a5f28-env-overrides" seLinuxMountContext=""
Oct 30 21:54:53.321871 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.321713    6133 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="484e37a0-fd1d-4fd0-a200-939b8945984b" volumeName="kubernetes.io/configmap/484e37a0-fd1d-4fd0-a200-939b8945984b-metrics-client-ca" seLinuxMountContext=""
Oct 30 21:54:53.321871 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.321739    6133 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="613f514c-6011-4936-8df5-a59b960cd3a6" volumeName="kubernetes.io/configmap/613f514c-6011-4936-8df5-a59b960cd3a6-iptables-alerter-script" seLinuxMountContext=""
Oct 30 21:54:53.321871 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.321754    6133 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="bebdb584-1a96-453d-8a6d-b332e47bec3d" volumeName="kubernetes.io/empty-dir/bebdb584-1a96-453d-8a6d-b332e47bec3d-etc-tuned" seLinuxMountContext=""
Oct 30 21:54:53.321871 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.321767    6133 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="fbd8f09b-9962-4910-8b25-267cbbb3ea5d" volumeName="kubernetes.io/secret/fbd8f09b-9962-4910-8b25-267cbbb3ea5d-secret-thanos-querier-kube-rbac-proxy-metrics" seLinuxMountContext=""
Oct 30 21:54:53.321871 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.321781    6133 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="fc4a67d9-8c7c-4f71-9a8a-2bba09329bc6" volumeName="kubernetes.io/projected/fc4a67d9-8c7c-4f71-9a8a-2bba09329bc6-kube-api-access-jp25c" seLinuxMountContext=""
Oct 30 21:54:53.321871 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.321793    6133 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="209b4e58-e6c2-48f1-8589-f7b6be79720b" volumeName="kubernetes.io/empty-dir/209b4e58-e6c2-48f1-8589-f7b6be79720b-tmp-dir" seLinuxMountContext=""
Oct 30 21:54:53.321871 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.321806    6133 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="209b4e58-e6c2-48f1-8589-f7b6be79720b" volumeName="kubernetes.io/secret/209b4e58-e6c2-48f1-8589-f7b6be79720b-metrics-tls" seLinuxMountContext=""
Oct 30 21:54:53.321871 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.321818    6133 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="bd93518f-2f96-4483-9e92-513109dd37cb" volumeName="kubernetes.io/secret/bd93518f-2f96-4483-9e92-513109dd37cb-secret-metrics-client-certs" seLinuxMountContext=""
Oct 30 21:54:53.321871 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.321830    6133 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="bebdb584-1a96-453d-8a6d-b332e47bec3d" volumeName="kubernetes.io/projected/bebdb584-1a96-453d-8a6d-b332e47bec3d-kube-api-access-r4qz4" seLinuxMountContext=""
Oct 30 21:54:53.321871 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.321844    6133 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="fbd8f09b-9962-4910-8b25-267cbbb3ea5d" volumeName="kubernetes.io/empty-dir/fbd8f09b-9962-4910-8b25-267cbbb3ea5d-tmp" seLinuxMountContext=""
Oct 30 21:54:53.321871 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.321855    6133 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="bd93518f-2f96-4483-9e92-513109dd37cb" volumeName="kubernetes.io/projected/bd93518f-2f96-4483-9e92-513109dd37cb-tls-assets" seLinuxMountContext=""
Oct 30 21:54:53.321871 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.321866    6133 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="bd93518f-2f96-4483-9e92-513109dd37cb" volumeName="kubernetes.io/secret/bd93518f-2f96-4483-9e92-513109dd37cb-web-config" seLinuxMountContext=""
Oct 30 21:54:53.321871 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.321879    6133 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="fbd8f09b-9962-4910-8b25-267cbbb3ea5d" volumeName="kubernetes.io/secret/fbd8f09b-9962-4910-8b25-267cbbb3ea5d-secret-thanos-querier-kube-rbac-proxy-rules" seLinuxMountContext=""
Oct 30 21:54:53.321871 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.321898    6133 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="fbd8f09b-9962-4910-8b25-267cbbb3ea5d" volumeName="kubernetes.io/secret/fbd8f09b-9962-4910-8b25-267cbbb3ea5d-secret-grpc-tls" seLinuxMountContext=""
Oct 30 21:54:53.323743 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.321912    6133 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="fe472529-1294-46a1-a666-dc00d5d37eca" volumeName="kubernetes.io/projected/fe472529-1294-46a1-a666-dc00d5d37eca-kube-api-access-nbqv6" seLinuxMountContext=""
Oct 30 21:54:53.323743 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.321929    6133 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="209b4e58-e6c2-48f1-8589-f7b6be79720b" volumeName="kubernetes.io/configmap/209b4e58-e6c2-48f1-8589-f7b6be79720b-config-volume" seLinuxMountContext=""
Oct 30 21:54:53.323743 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.321942    6133 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="446beacf-70a0-4d47-96ea-bbe1ca4a5f28" volumeName="kubernetes.io/configmap/446beacf-70a0-4d47-96ea-bbe1ca4a5f28-ovnkube-script-lib" seLinuxMountContext=""
Oct 30 21:54:53.323743 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.321956    6133 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="6192d2be-bba7-4099-b54c-e7e9b8d75389" volumeName="kubernetes.io/configmap/6192d2be-bba7-4099-b54c-e7e9b8d75389-kube-rbac-proxy-cm" seLinuxMountContext=""
Oct 30 21:54:53.323743 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.321970    6133 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="bd93518f-2f96-4483-9e92-513109dd37cb" volumeName="kubernetes.io/secret/bd93518f-2f96-4483-9e92-513109dd37cb-secret-prometheus-k8s-tls" seLinuxMountContext=""
Oct 30 21:54:53.323743 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.321983    6133 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="bd93518f-2f96-4483-9e92-513109dd37cb" volumeName="kubernetes.io/secret/bd93518f-2f96-4483-9e92-513109dd37cb-thanos-prometheus-http-client-file" seLinuxMountContext=""
Oct 30 21:54:53.323743 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.322020    6133 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="fbd8f09b-9962-4910-8b25-267cbbb3ea5d" volumeName="kubernetes.io/configmap/fbd8f09b-9962-4910-8b25-267cbbb3ea5d-metrics-client-ca" seLinuxMountContext=""
Oct 30 21:54:53.323743 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.322032    6133 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="fc4a67d9-8c7c-4f71-9a8a-2bba09329bc6" volumeName="kubernetes.io/empty-dir/fc4a67d9-8c7c-4f71-9a8a-2bba09329bc6-tmp-dir" seLinuxMountContext=""
Oct 30 21:54:53.323743 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.322044    6133 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="bd93518f-2f96-4483-9e92-513109dd37cb" volumeName="kubernetes.io/configmap/bd93518f-2f96-4483-9e92-513109dd37cb-prometheus-trusted-ca-bundle" seLinuxMountContext=""
Oct 30 21:54:53.323743 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.322055    6133 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="17da628d-31f5-456c-ad37-52c884d22cf4" volumeName="kubernetes.io/configmap/17da628d-31f5-456c-ad37-52c884d22cf4-metrics-client-ca" seLinuxMountContext=""
Oct 30 21:54:53.323743 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.322070    6133 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="32ae0d42-4b76-47b7-b9d6-26b09f00ba5d" volumeName="kubernetes.io/projected/32ae0d42-4b76-47b7-b9d6-26b09f00ba5d-kube-api-access-ljsqq" seLinuxMountContext=""
Oct 30 21:54:53.323743 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.322079    6133 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3e60fceb-8682-45cc-a472-f3d7017c3395" volumeName="kubernetes.io/configmap/3e60fceb-8682-45cc-a472-f3d7017c3395-whereabouts-flatfile-configmap" seLinuxMountContext=""
Oct 30 21:54:53.323743 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.322097    6133 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="9d2cced5-9ceb-421f-b831-8370a5ee0ca9" volumeName="kubernetes.io/projected/9d2cced5-9ceb-421f-b831-8370a5ee0ca9-kube-api-access-w78hr" seLinuxMountContext=""
Oct 30 21:54:53.323743 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.322105    6133 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="bd93518f-2f96-4483-9e92-513109dd37cb" volumeName="kubernetes.io/secret/bd93518f-2f96-4483-9e92-513109dd37cb-secret-prometheus-k8s-kube-rbac-proxy-web" seLinuxMountContext=""
Oct 30 21:54:53.323743 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.322115    6133 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="af83b188-09cb-4621-a9da-be33856d19f0" volumeName="kubernetes.io/secret/af83b188-09cb-4621-a9da-be33856d19f0-original-pull-secret" seLinuxMountContext=""
Oct 30 21:54:53.323743 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.322122    6133 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="484e37a0-fd1d-4fd0-a200-939b8945984b" volumeName="kubernetes.io/secret/484e37a0-fd1d-4fd0-a200-939b8945984b-openshift-state-metrics-kube-rbac-proxy-config" seLinuxMountContext=""
Oct 30 21:54:53.323743 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.322130    6133 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="532658b6-c86e-4feb-89cb-14a91316a7a0" volumeName="kubernetes.io/projected/532658b6-c86e-4feb-89cb-14a91316a7a0-kube-api-access-r6wtm" seLinuxMountContext=""
Oct 30 21:54:53.323743 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.322138    6133 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="5d5057fc-dd8a-4dce-b8bb-70b2dc3b074c" volumeName="kubernetes.io/configmap/5d5057fc-dd8a-4dce-b8bb-70b2dc3b074c-multus-daemon-config" seLinuxMountContext=""
Oct 30 21:54:53.323743 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.322147    6133 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="5d5057fc-dd8a-4dce-b8bb-70b2dc3b074c" volumeName="kubernetes.io/configmap/5d5057fc-dd8a-4dce-b8bb-70b2dc3b074c-cni-binary-copy" seLinuxMountContext=""
Oct 30 21:54:53.323743 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.322155    6133 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="9d2cced5-9ceb-421f-b831-8370a5ee0ca9" volumeName="kubernetes.io/secret/9d2cced5-9ceb-421f-b831-8370a5ee0ca9-monitoring-plugin-cert" seLinuxMountContext=""
Oct 30 21:54:53.323743 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.322162    6133 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="15e1de31-92a1-4179-8ec1-400346880d1d" volumeName="kubernetes.io/projected/15e1de31-92a1-4179-8ec1-400346880d1d-kube-api-access-lgk4w" seLinuxMountContext=""
Oct 30 21:54:53.323743 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.322170    6133 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="209b4e58-e6c2-48f1-8589-f7b6be79720b" volumeName="kubernetes.io/projected/209b4e58-e6c2-48f1-8589-f7b6be79720b-kube-api-access-mrtmr" seLinuxMountContext=""
Oct 30 21:54:53.323743 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.322177    6133 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4621048c-230e-441f-85fc-423e4ba525ec" volumeName="kubernetes.io/projected/4621048c-230e-441f-85fc-423e4ba525ec-kube-api-access-9f2kg" seLinuxMountContext=""
Oct 30 21:54:53.323743 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.322185    6133 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="484e37a0-fd1d-4fd0-a200-939b8945984b" volumeName="kubernetes.io/secret/484e37a0-fd1d-4fd0-a200-939b8945984b-openshift-state-metrics-tls" seLinuxMountContext=""
Oct 30 21:54:53.323743 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.322195    6133 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7ddfa67d-20a0-473d-937f-25fd8e0ba80e" volumeName="kubernetes.io/secret/7ddfa67d-20a0-473d-937f-25fd8e0ba80e-node-exporter-tls" seLinuxMountContext=""
Oct 30 21:54:53.323743 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.322203    6133 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="bebdb584-1a96-453d-8a6d-b332e47bec3d" volumeName="kubernetes.io/empty-dir/bebdb584-1a96-453d-8a6d-b332e47bec3d-tmp" seLinuxMountContext=""
Oct 30 21:54:53.323743 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.322210    6133 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="2bcee03a-9b40-4155-8d81-43dece439162" volumeName="kubernetes.io/configmap/2bcee03a-9b40-4155-8d81-43dece439162-konnectivity-ca" seLinuxMountContext=""
Oct 30 21:54:53.323743 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.322223    6133 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3e60fceb-8682-45cc-a472-f3d7017c3395" volumeName="kubernetes.io/configmap/3e60fceb-8682-45cc-a472-f3d7017c3395-cni-binary-copy" seLinuxMountContext=""
Oct 30 21:54:53.323743 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.322231    6133 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3e60fceb-8682-45cc-a472-f3d7017c3395" volumeName="kubernetes.io/projected/3e60fceb-8682-45cc-a472-f3d7017c3395-kube-api-access-6bsvp" seLinuxMountContext=""
Oct 30 21:54:53.323743 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.322238    6133 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="5d5057fc-dd8a-4dce-b8bb-70b2dc3b074c" volumeName="kubernetes.io/projected/5d5057fc-dd8a-4dce-b8bb-70b2dc3b074c-kube-api-access-kncjh" seLinuxMountContext=""
Oct 30 21:54:53.323743 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.322246    6133 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="613f514c-6011-4936-8df5-a59b960cd3a6" volumeName="kubernetes.io/projected/613f514c-6011-4936-8df5-a59b960cd3a6-kube-api-access-h6b6r" seLinuxMountContext=""
Oct 30 21:54:53.323743 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.322253    6133 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7ddfa67d-20a0-473d-937f-25fd8e0ba80e" volumeName="kubernetes.io/configmap/7ddfa67d-20a0-473d-937f-25fd8e0ba80e-node-exporter-accelerators-collector-config" seLinuxMountContext=""
Oct 30 21:54:53.323743 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.322261    6133 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="bd93518f-2f96-4483-9e92-513109dd37cb" volumeName="kubernetes.io/configmap/bd93518f-2f96-4483-9e92-513109dd37cb-prometheus-k8s-rulefiles-2" seLinuxMountContext=""
Oct 30 21:54:53.323743 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.322269    6133 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="bd93518f-2f96-4483-9e92-513109dd37cb" volumeName="kubernetes.io/secret/bd93518f-2f96-4483-9e92-513109dd37cb-secret-kube-rbac-proxy" seLinuxMountContext=""
Oct 30 21:54:53.323743 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.322276    6133 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="446beacf-70a0-4d47-96ea-bbe1ca4a5f28" volumeName="kubernetes.io/projected/446beacf-70a0-4d47-96ea-bbe1ca4a5f28-kube-api-access-mkrsw" seLinuxMountContext=""
Oct 30 21:54:53.323743 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.322284    6133 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="484e37a0-fd1d-4fd0-a200-939b8945984b" volumeName="kubernetes.io/projected/484e37a0-fd1d-4fd0-a200-939b8945984b-kube-api-access-n87x8" seLinuxMountContext=""
Oct 30 21:54:53.323743 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.322292    6133 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="6192d2be-bba7-4099-b54c-e7e9b8d75389" volumeName="kubernetes.io/projected/6192d2be-bba7-4099-b54c-e7e9b8d75389-kube-api-access-jw4wk" seLinuxMountContext=""
Oct 30 21:54:53.323743 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.322300    6133 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="bd93518f-2f96-4483-9e92-513109dd37cb" volumeName="kubernetes.io/projected/bd93518f-2f96-4483-9e92-513109dd37cb-kube-api-access-qrt69" seLinuxMountContext=""
Oct 30 21:54:53.323743 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.322307    6133 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7ddfa67d-20a0-473d-937f-25fd8e0ba80e" volumeName="kubernetes.io/projected/7ddfa67d-20a0-473d-937f-25fd8e0ba80e-kube-api-access-wsnr5" seLinuxMountContext=""
Oct 30 21:54:53.323743 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.322314    6133 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="bd93518f-2f96-4483-9e92-513109dd37cb" volumeName="kubernetes.io/configmap/bd93518f-2f96-4483-9e92-513109dd37cb-configmap-metrics-client-ca" seLinuxMountContext=""
Oct 30 21:54:53.323743 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.322321    6133 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="fbd8f09b-9962-4910-8b25-267cbbb3ea5d" volumeName="kubernetes.io/secret/fbd8f09b-9962-4910-8b25-267cbbb3ea5d-secret-thanos-querier-kube-rbac-proxy" seLinuxMountContext=""
Oct 30 21:54:53.323743 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.322329    6133 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="17da628d-31f5-456c-ad37-52c884d22cf4" volumeName="kubernetes.io/secret/17da628d-31f5-456c-ad37-52c884d22cf4-kube-state-metrics-kube-rbac-proxy-config" seLinuxMountContext=""
Oct 30 21:54:53.323743 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.322338    6133 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="6192d2be-bba7-4099-b54c-e7e9b8d75389" volumeName="kubernetes.io/empty-dir/6192d2be-bba7-4099-b54c-e7e9b8d75389-data-volume" seLinuxMountContext=""
Oct 30 21:54:53.323743 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.322346    6133 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="bd93518f-2f96-4483-9e92-513109dd37cb" volumeName="kubernetes.io/configmap/bd93518f-2f96-4483-9e92-513109dd37cb-prometheus-k8s-rulefiles-0" seLinuxMountContext=""
Oct 30 21:54:53.323743 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.322354    6133 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="bd93518f-2f96-4483-9e92-513109dd37cb" volumeName="kubernetes.io/empty-dir/bd93518f-2f96-4483-9e92-513109dd37cb-prometheus-k8s-db" seLinuxMountContext=""
Oct 30 21:54:53.323743 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.322362    6133 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="bd93518f-2f96-4483-9e92-513109dd37cb" volumeName="kubernetes.io/secret/bd93518f-2f96-4483-9e92-513109dd37cb-config" seLinuxMountContext=""
Oct 30 21:54:53.323743 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.322370    6133 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="fbd8f09b-9962-4910-8b25-267cbbb3ea5d" volumeName="kubernetes.io/secret/fbd8f09b-9962-4910-8b25-267cbbb3ea5d-secret-thanos-querier-tls" seLinuxMountContext=""
Oct 30 21:54:53.323743 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.322380    6133 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="bd93518f-2f96-4483-9e92-513109dd37cb" volumeName="kubernetes.io/configmap/bd93518f-2f96-4483-9e92-513109dd37cb-prometheus-k8s-rulefiles-1" seLinuxMountContext=""
Oct 30 21:54:53.323743 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.322389    6133 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3e60fceb-8682-45cc-a472-f3d7017c3395" volumeName="kubernetes.io/configmap/3e60fceb-8682-45cc-a472-f3d7017c3395-cni-sysctl-allowlist" seLinuxMountContext=""
Oct 30 21:54:53.323743 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.322397    6133 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="446beacf-70a0-4d47-96ea-bbe1ca4a5f28" volumeName="kubernetes.io/configmap/446beacf-70a0-4d47-96ea-bbe1ca4a5f28-ovnkube-config" seLinuxMountContext=""
Oct 30 21:54:53.323743 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.322405    6133 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="446beacf-70a0-4d47-96ea-bbe1ca4a5f28" volumeName="kubernetes.io/secret/446beacf-70a0-4d47-96ea-bbe1ca4a5f28-ovn-node-metrics-cert" seLinuxMountContext=""
Oct 30 21:54:53.323743 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.322414    6133 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4621048c-230e-441f-85fc-423e4ba525ec" volumeName="kubernetes.io/secret/4621048c-230e-441f-85fc-423e4ba525ec-cert" seLinuxMountContext=""
Oct 30 21:54:53.323743 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.322422    6133 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7ddfa67d-20a0-473d-937f-25fd8e0ba80e" volumeName="kubernetes.io/empty-dir/7ddfa67d-20a0-473d-937f-25fd8e0ba80e-node-exporter-textfile" seLinuxMountContext=""
Oct 30 21:54:53.323743 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.322429    6133 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="fbd8f09b-9962-4910-8b25-267cbbb3ea5d" volumeName="kubernetes.io/secret/fbd8f09b-9962-4910-8b25-267cbbb3ea5d-secret-thanos-querier-kube-rbac-proxy-web" seLinuxMountContext=""
Oct 30 21:54:53.323743 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.322438    6133 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="bd93518f-2f96-4483-9e92-513109dd37cb" volumeName="kubernetes.io/configmap/bd93518f-2f96-4483-9e92-513109dd37cb-configmap-kubelet-serving-ca-bundle" seLinuxMountContext=""
Oct 30 21:54:53.323743 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.322445    6133 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="6192d2be-bba7-4099-b54c-e7e9b8d75389" volumeName="kubernetes.io/secret/6192d2be-bba7-4099-b54c-e7e9b8d75389-insights-runtime-extractor-tls" seLinuxMountContext=""
Oct 30 21:54:53.323743 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.322454    6133 reconstruct.go:97] "Volume reconstruction finished"
Oct 30 21:54:53.323743 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.322460    6133 reconciler.go:29] "Reconciler: start to sync state"
Oct 30 21:54:53.348299 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.348254    6133 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Oct 30 21:54:53.349781 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.349755    6133 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Oct 30 21:54:53.349781 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.349782    6133 status_manager.go:249] "Starting to sync pod status with apiserver"
Oct 30 21:54:53.351029 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.349807    6133 kubelet.go:2521] "Starting kubelet main sync loop"
Oct 30 21:54:53.351029 ip-10-0-143-246 kubenswrapper[6133]: E1030 21:54:53.349856    6133 kubelet.go:2545] "Skipping pod synchronization" err="[container runtime status check may not have completed yet, PLEG is not healthy: pleg has yet to be successful]"
Oct 30 21:54:53.362601 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.362556    6133 generic.go:356] "Generic (PLEG): container finished" podID="7ddfa67d-20a0-473d-937f-25fd8e0ba80e" containerID="fd1e33dbc11f4190c42b506a8dcb51ecc84c5ff25b26b988b4b833e2b1b50381" exitCode=0
Oct 30 21:54:53.364624 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.364601    6133 reconstruct.go:205] "DevicePaths of reconstructed volumes updated"
Oct 30 21:54:53.366133 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.366107    6133 reflector.go:446] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 21:54:53.366230 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.366152    6133 reflector.go:446] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 21:54:53.370409 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.370382    6133 generic.go:356] "Generic (PLEG): container finished" podID="3e60fceb-8682-45cc-a472-f3d7017c3395" containerID="3fb85fa86680ca79432d7d50189dc1d7161885943aac070e88bd75560ad8777a" exitCode=0
Oct 30 21:54:53.370409 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.370408    6133 generic.go:356] "Generic (PLEG): container finished" podID="3e60fceb-8682-45cc-a472-f3d7017c3395" containerID="524aa4acadbe157d10a1ca4405db94dbf41a555bf00c84ccec93c047e8e91d67" exitCode=0
Oct 30 21:54:53.370531 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.370419    6133 generic.go:356] "Generic (PLEG): container finished" podID="3e60fceb-8682-45cc-a472-f3d7017c3395" containerID="73754faf5ede0d182282006d5473437e56d30bd3321891a80c51f6031129545c" exitCode=0
Oct 30 21:54:53.370531 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.370429    6133 generic.go:356] "Generic (PLEG): container finished" podID="3e60fceb-8682-45cc-a472-f3d7017c3395" containerID="7d2403a8c6e18c74faedb7c8c33fe7def1f92797a71523a996fd29c647c6778d" exitCode=0
Oct 30 21:54:53.370531 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.370438    6133 generic.go:356] "Generic (PLEG): container finished" podID="3e60fceb-8682-45cc-a472-f3d7017c3395" containerID="4dfa29366f9ad760161c09fb04efa3a6c62b648040fc26de207d36df85a10381" exitCode=0
Oct 30 21:54:53.370531 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.370447    6133 generic.go:356] "Generic (PLEG): container finished" podID="3e60fceb-8682-45cc-a472-f3d7017c3395" containerID="03a8dd6b8d585e7683860d1e3cd1375f2bf266003d39ed2d5ffccabf4eb09c75" exitCode=0
Oct 30 21:54:53.382464 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.382432    6133 generic.go:356] "Generic (PLEG): container finished" podID="bd93518f-2f96-4483-9e92-513109dd37cb" containerID="1afbd58205aa89f01dece6ea4aceecdb2c1a95d6f90583bfa5c329026ead89df" exitCode=0
Oct 30 21:54:53.389276 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.389246    6133 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 21:54:53.389440 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.389313    6133 reflector.go:446] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 21:54:53.405783 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.405744    6133 generic.go:356] "Generic (PLEG): container finished" podID="b7eadc83c6b2f891221807815b7cecbc" containerID="067e439d4a1c32de3c77bcd42a8c171531377446c38f295e1e031ce55133e5a1" exitCode=0
Oct 30 21:54:53.435027 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.434938    6133 manager.go:324] Recovery completed
Oct 30 21:54:53.450498 ip-10-0-143-246 kubenswrapper[6133]: E1030 21:54:53.450463    6133 kubelet.go:2545] "Skipping pod synchronization" err="container runtime status check may not have completed yet"
Oct 30 21:54:53.459095 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.459063    6133 cpu_manager.go:226] "Starting" policy="none"
Oct 30 21:54:53.459095 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.459094    6133 cpu_manager.go:227] "Reconciling" reconcilePeriod="10s"
Oct 30 21:54:53.459276 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.459122    6133 state_mem.go:41] "Initialized" logger="CPUManager state checkpoint.CPUManager state memory"
Oct 30 21:54:53.459339 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.459320    6133 state_mem.go:94] "Updated default CPUSet" logger="CPUManager state checkpoint.CPUManager state memory" cpuSet=""
Oct 30 21:54:53.459382 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.459335    6133 state_mem.go:102] "Updated CPUSet assignments" logger="CPUManager state checkpoint.CPUManager state memory" assignments={}
Oct 30 21:54:53.459382 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.459360    6133 state_checkpoint.go:142] "restored state from checkpoint" logger="CPUManager state checkpoint"
Oct 30 21:54:53.459382 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.459369    6133 state_checkpoint.go:143] "defaultCPUSet" logger="CPUManager state checkpoint" defaultCpuSet=""
Oct 30 21:54:53.459382 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.459375    6133 policy_none.go:50] "Start"
Oct 30 21:54:53.459553 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.459399    6133 memory_manager.go:187] "Starting memorymanager" policy="None"
Oct 30 21:54:53.459553 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.459412    6133 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Oct 30 21:54:53.459638 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.459626    6133 state_mem.go:77] "Updated machine memory state" logger="Memory Manager state checkpoint"
Oct 30 21:54:53.459687 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.459637    6133 state_checkpoint.go:85] "State checkpoint: restored state from checkpoint" logger="Memory Manager state checkpoint"
Oct 30 21:54:53.459687 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.459645    6133 policy_none.go:44] "Start"
Oct 30 21:54:53.464725 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.464688    6133 node_container_manager_linux.go:130] "Enforcing system reserved compressible on cgroup" cgroupName="/system.slice" limits={"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"}
Oct 30 21:54:53.468537 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.468521    6133 manager.go:341] "Starting Device Plugin manager"
Oct 30 21:54:53.469043 ip-10-0-143-246 kubenswrapper[6133]: E1030 21:54:53.468554    6133 manager.go:525] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Oct 30 21:54:53.469122 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.469046    6133 server.go:87] "Starting device plugin registration server"
Oct 30 21:54:53.469359 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.469345    6133 eviction_manager.go:194] "Eviction manager: starting control loop"
Oct 30 21:54:53.469436 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.469360    6133 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Oct 30 21:54:53.469482 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.469452    6133 plugin_watcher.go:53] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Oct 30 21:54:53.469631 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.469579    6133 plugin_watcher.go:201] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Oct 30 21:54:53.469740 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.469650    6133 plugin_manager.go:119] "The desired_state_of_world populator (plugin watcher) starts"
Oct 30 21:54:53.469740 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.469659    6133 plugin_manager.go:121] "Starting Kubelet Plugin Manager"
Oct 30 21:54:53.469826 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.469726    6133 reconciler.go:167] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-10-30T21:54:53.469628066Z","UUID":"a593604c-f2d6-45f6-ab9f-c37975bac879","Handler":null,"Name":"","Endpoint":""}
Oct 30 21:54:53.471169 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.471152    6133 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Oct 30 21:54:53.471267 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.471188    6133 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Oct 30 21:54:53.471384 ip-10-0-143-246 kubenswrapper[6133]: E1030 21:54:53.471345    6133 eviction_manager.go:272] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Oct 30 21:54:53.569882 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.569835    6133 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 21:54:53.571843 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.571825    6133 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-143-246.ec2.internal" event="NodeHasSufficientMemory"
Oct 30 21:54:53.571915 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.571855    6133 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-143-246.ec2.internal" event="NodeHasNoDiskPressure"
Oct 30 21:54:53.571915 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.571866    6133 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-143-246.ec2.internal" event="NodeHasSufficientPID"
Oct 30 21:54:53.571915 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.571907    6133 kubelet_node_status.go:77] "Attempting to register node" node="ip-10-0-143-246.ec2.internal"
Oct 30 21:54:53.610484 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.610456    6133 kubelet_node_status.go:126] "Node was previously registered" node="ip-10-0-143-246.ec2.internal"
Oct 30 21:54:53.610634 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.610546    6133 kubelet_node_status.go:80] "Successfully registered node" node="ip-10-0-143-246.ec2.internal"
Oct 30 21:54:53.651301 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.651233    6133 kubelet.go:2608] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-246.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-143-246.ec2.internal"]
Oct 30 21:54:53.651966 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.651945    6133 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-246.ec2.internal"
Oct 30 21:54:53.652049 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.651980    6133 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-246.ec2.internal"
Oct 30 21:54:53.652101 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.652019    6133 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-246.ec2.internal" event={"ID":"b29468cdb315509fa1bb87ee196f0663","Type":"ContainerStarted","Data":"3979d411383ce88f5fdefdffe5a734be0e811ea8fac3495a5a575f36fd3e01b1"}
Oct 30 21:54:53.652101 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.652087    6133 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-246.ec2.internal" event={"ID":"b29468cdb315509fa1bb87ee196f0663","Type":"ContainerStarted","Data":"a3f130547617214efc2a1c06555379109c660bad2501f887be595e3d279ea8a4"}
Oct 30 21:54:53.652215 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.652109    6133 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-246.ec2.internal" event={"ID":"b7eadc83c6b2f891221807815b7cecbc","Type":"ContainerStarted","Data":"d372e3237f24097aed64d1f703399768a6d1994f3c7834ff64f75de0dd9df31c"}
Oct 30 21:54:53.652215 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.652130    6133 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-246.ec2.internal" event={"ID":"b7eadc83c6b2f891221807815b7cecbc","Type":"ContainerDied","Data":"067e439d4a1c32de3c77bcd42a8c171531377446c38f295e1e031ce55133e5a1"}
Oct 30 21:54:53.652215 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.652151    6133 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-246.ec2.internal" event={"ID":"b7eadc83c6b2f891221807815b7cecbc","Type":"ContainerStarted","Data":"a1a23f1cc1936fcf6f0fbc3af4009a0cd4a17790ed28b13c1dfc5accffe8f597"}
Oct 30 21:54:53.682632 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.682603    6133 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 30 21:54:53.682794 ip-10-0-143-246 kubenswrapper[6133]: E1030 21:54:53.682658    6133 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-apiserver-proxy-ip-10-0-143-246.ec2.internal\" already exists" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-246.ec2.internal"
Oct 30 21:54:53.683405 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.683392    6133 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 30 21:54:53.683448 ip-10-0-143-246 kubenswrapper[6133]: E1030 21:54:53.683427    6133 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-rbac-proxy-crio-ip-10-0-143-246.ec2.internal\" already exists" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-246.ec2.internal"
Oct 30 21:54:53.766792 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.766753    6133 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/b7eadc83c6b2f891221807815b7cecbc-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-246.ec2.internal\" (UID: \"b7eadc83c6b2f891221807815b7cecbc\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-246.ec2.internal"
Oct 30 21:54:53.766912 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.766795    6133 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b7eadc83c6b2f891221807815b7cecbc-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-246.ec2.internal\" (UID: \"b7eadc83c6b2f891221807815b7cecbc\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-246.ec2.internal"
Oct 30 21:54:53.766912 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.766821    6133 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/b29468cdb315509fa1bb87ee196f0663-config\") pod \"kube-apiserver-proxy-ip-10-0-143-246.ec2.internal\" (UID: \"b29468cdb315509fa1bb87ee196f0663\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-246.ec2.internal"
Oct 30 21:54:53.867364 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.867330    6133 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/b29468cdb315509fa1bb87ee196f0663-config\") pod \"kube-apiserver-proxy-ip-10-0-143-246.ec2.internal\" (UID: \"b29468cdb315509fa1bb87ee196f0663\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-246.ec2.internal"
Oct 30 21:54:53.867364 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.867364    6133 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/b7eadc83c6b2f891221807815b7cecbc-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-246.ec2.internal\" (UID: \"b7eadc83c6b2f891221807815b7cecbc\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-246.ec2.internal"
Oct 30 21:54:53.867570 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.867383    6133 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b7eadc83c6b2f891221807815b7cecbc-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-246.ec2.internal\" (UID: \"b7eadc83c6b2f891221807815b7cecbc\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-246.ec2.internal"
Oct 30 21:54:53.867570 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.867449    6133 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/b29468cdb315509fa1bb87ee196f0663-config\") pod \"kube-apiserver-proxy-ip-10-0-143-246.ec2.internal\" (UID: \"b29468cdb315509fa1bb87ee196f0663\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-246.ec2.internal"
Oct 30 21:54:53.867570 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.867526    6133 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b7eadc83c6b2f891221807815b7cecbc-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-246.ec2.internal\" (UID: \"b7eadc83c6b2f891221807815b7cecbc\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-246.ec2.internal"
Oct 30 21:54:53.867661 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:53.867566    6133 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/b7eadc83c6b2f891221807815b7cecbc-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-246.ec2.internal\" (UID: \"b7eadc83c6b2f891221807815b7cecbc\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-246.ec2.internal"
Oct 30 21:54:54.304968 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.304937    6133 apiserver.go:52] "Watching apiserver"
Oct 30 21:54:54.360724 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.360691    6133 reflector.go:446] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Oct 30 21:54:54.361403 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.361378    6133 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-77f4f76594-htsvq","openshift-network-operator/iptables-alerter-kghfh","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-246.ec2.internal","openshift-monitoring/kube-state-metrics-5dc8d7d89b-8727r","openshift-multus/network-metrics-daemon-n8xrh","openshift-monitoring/node-exporter-df5jt","openshift-network-diagnostics/network-check-target-fvbz8","openshift-ovn-kubernetes/ovnkube-node-mn9vl","kube-system/global-pull-secret-syncer-q7fwz","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kqbp9","openshift-monitoring/openshift-state-metrics-7bd7489957-f26w7","openshift-cluster-node-tuning-operator/tuned-h8csb","openshift-multus/multus-jkdxt","kube-system/kube-apiserver-proxy-ip-10-0-143-246.ec2.internal","openshift-image-registry/node-ca-qzgkj","openshift-insights/insights-runtime-extractor-6jppj","openshift-monitoring/thanos-querier-868d6f78df-nx2k6","openshift-monitoring/prometheus-k8s-0","openshift-multus/multus-additional-cni-plugins-c9tnl","kube-system/konnectivity-agent-thksb","openshift-dns/node-resolver-wwb9f","openshift-ingress-canary/ingress-canary-fl5mx","openshift-dns/dns-default-fqmbc"]
Oct 30 21:54:54.380660 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.380627    6133 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Oct 30 21:54:54.380660 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.380660    6133 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Oct 30 21:54:54.380938 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.380757    6133 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Oct 30 21:54:54.380938 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.380768    6133 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Oct 30 21:54:54.380938 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.380793    6133 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Oct 30 21:54:54.380938 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.380806    6133 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Oct 30 21:54:54.380938 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.380839    6133 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Oct 30 21:54:54.380938 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.380860    6133 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Oct 30 21:54:54.380938 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.380915    6133 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Oct 30 21:54:54.381521 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.381490    6133 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent-dockercfg-gzm2w\""
Oct 30 21:54:54.381696 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.381682    6133 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Oct 30 21:54:54.381842 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.381827    6133 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Oct 30 21:54:54.381892 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.381883    6133 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Oct 30 21:54:54.382002 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.381914    6133 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-check-target-dockercfg-vwffm\""
Oct 30 21:54:54.382002 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.381918    6133 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-2n5cw\""
Oct 30 21:54:54.382002 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.381948    6133 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-vv6qp\""
Oct 30 21:54:54.382262 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.382051    6133 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-cn5n2\""
Oct 30 21:54:54.382262 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.382121    6133 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Oct 30 21:54:54.382262 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.382124    6133 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Oct 30 21:54:54.382262 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.382155    6133 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Oct 30 21:54:54.382428 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.382274    6133 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Oct 30 21:54:54.382428 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.382288    6133 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Oct 30 21:54:54.382428 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.382332    6133 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Oct 30 21:54:54.382428 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.382353    6133 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Oct 30 21:54:54.382428 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.382383    6133 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"global-pull-secret-syncer-dockercfg-cggx8\""
Oct 30 21:54:54.390240 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.390190    6133 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Oct 30 21:54:54.390240 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.390218    6133 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Oct 30 21:54:54.390240 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.390230    6133 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Oct 30 21:54:54.390552 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.390447    6133 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Oct 30 21:54:54.390552 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.390454    6133 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Oct 30 21:54:54.390552 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.390489    6133 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-8hlnq\""
Oct 30 21:54:54.390552 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.390501    6133 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Oct 30 21:54:54.390552 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.390534    6133 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Oct 30 21:54:54.390784 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.390626    6133 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-htkkz\""
Oct 30 21:54:54.390784 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.390649    6133 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Oct 30 21:54:54.390784 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.390660    6133 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Oct 30 21:54:54.390784 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.390693    6133 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-nhqdj\""
Oct 30 21:54:54.390784 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.390711    6133 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-v8mh7\""
Oct 30 21:54:54.390784 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.390719    6133 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Oct 30 21:54:54.390784 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.390712    6133 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Oct 30 21:54:54.390784 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.390722    6133 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Oct 30 21:54:54.390784 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.390755    6133 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-28tsh\""
Oct 30 21:54:54.391107 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.390793    6133 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Oct 30 21:54:54.391107 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.390814    6133 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"ingress-canary-dockercfg-5mpht\""
Oct 30 21:54:54.391107 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.390879    6133 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-hxntv\""
Oct 30 21:54:54.395476 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.395446    6133 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\""
Oct 30 21:54:54.396171 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.396148    6133 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\""
Oct 30 21:54:54.396267 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.396244    6133 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Oct 30 21:54:54.396577 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.396556    6133 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Oct 30 21:54:54.396657 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.396636    6133 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\""
Oct 30 21:54:54.396809 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.396797    6133 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\""
Oct 30 21:54:54.396845 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.396810    6133 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\""
Oct 30 21:54:54.396874 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.396854    6133 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\""
Oct 30 21:54:54.397312 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.397300    6133 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Oct 30 21:54:54.397347 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.397319    6133 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Oct 30 21:54:54.397347 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.397334    6133 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Oct 30 21:54:54.397440 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.397370    6133 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Oct 30 21:54:54.397440 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.397421    6133 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Oct 30 21:54:54.397534 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.397448    6133 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Oct 30 21:54:54.397534 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.397521    6133 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Oct 30 21:54:54.397625 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.397534    6133 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Oct 30 21:54:54.397625 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.397563    6133 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Oct 30 21:54:54.397625 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.397571    6133 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-node-identity-dockercfg-h8qq9\""
Oct 30 21:54:54.397625 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.397566    6133 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Oct 30 21:54:54.397625 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.397589    6133 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Oct 30 21:54:54.397625 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.397589    6133 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Oct 30 21:54:54.397625 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.397550    6133 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Oct 30 21:54:54.397953 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.397598    6133 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Oct 30 21:54:54.397953 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.397635    6133 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-bv47g\""
Oct 30 21:54:54.397953 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.397564    6133 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Oct 30 21:54:54.397953 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.397647    6133 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Oct 30 21:54:54.397953 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.397594    6133 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Oct 30 21:54:54.397953 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.397660    6133 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-5kd5n\""
Oct 30 21:54:54.397953 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.397753    6133 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-5cssp\""
Oct 30 21:54:54.397953 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.397763    6133 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Oct 30 21:54:54.397953 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.397775    6133 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\""
Oct 30 21:54:54.397953 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.397813    6133 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\""
Oct 30 21:54:54.397953 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.397882    6133 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-8c44nd500k9ok\""
Oct 30 21:54:54.397953 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.397889    6133 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\""
Oct 30 21:54:54.397953 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.397886    6133 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-l4wkb\""
Oct 30 21:54:54.397953 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.397959    6133 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Oct 30 21:54:54.401521 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.401508    6133 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-9k9apv9hqg1mm\""
Oct 30 21:54:54.401521 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.401510    6133 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Oct 30 21:54:54.401689 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.401675    6133 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Oct 30 21:54:54.401861 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.401845    6133 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-772zc\""
Oct 30 21:54:54.401917 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.401872    6133 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-xwsb9\""
Oct 30 21:54:54.401917 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.401885    6133 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Oct 30 21:54:54.402004 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.401917    6133 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\""
Oct 30 21:54:54.402004 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.401920    6133 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Oct 30 21:54:54.402004 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.401922    6133 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Oct 30 21:54:54.402004 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.401947    6133 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\""
Oct 30 21:54:54.402004 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.401961    6133 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-pfg22\""
Oct 30 21:54:54.402830 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.402817    6133 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Oct 30 21:54:54.408756 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.408736    6133 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Oct 30 21:54:54.409706 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.409693    6133 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Oct 30 21:54:54.416820 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.416788    6133 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-246.ec2.internal"
Oct 30 21:54:54.416919 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.416879    6133 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-246.ec2.internal"
Oct 30 21:54:54.420892 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.420872    6133 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Oct 30 21:54:54.441903 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.441870    6133 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 30 21:54:54.442073 ip-10-0-143-246 kubenswrapper[6133]: E1030 21:54:54.441934    6133 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-apiserver-proxy-ip-10-0-143-246.ec2.internal\" already exists" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-246.ec2.internal"
Oct 30 21:54:54.443165 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.443151    6133 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 30 21:54:54.443220 ip-10-0-143-246 kubenswrapper[6133]: E1030 21:54:54.443195    6133 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-rbac-proxy-crio-ip-10-0-143-246.ec2.internal\" already exists" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-246.ec2.internal"
Oct 30 21:54:54.470231 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.470193    6133 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/5d5057fc-dd8a-4dce-b8bb-70b2dc3b074c-host-var-lib-cni-multus\") pod \"multus-jkdxt\" (UID: \"5d5057fc-dd8a-4dce-b8bb-70b2dc3b074c\") " pod="openshift-multus/multus-jkdxt"
Oct 30 21:54:54.470231 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.470228    6133 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/af83b188-09cb-4621-a9da-be33856d19f0-dbus\") pod \"global-pull-secret-syncer-q7fwz\" (UID: \"af83b188-09cb-4621-a9da-be33856d19f0\") " pod="kube-system/global-pull-secret-syncer-q7fwz"
Oct 30 21:54:54.470434 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.470249    6133 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/613f514c-6011-4936-8df5-a59b960cd3a6-iptables-alerter-script\") pod \"iptables-alerter-kghfh\" (UID: \"613f514c-6011-4936-8df5-a59b960cd3a6\") " pod="openshift-network-operator/iptables-alerter-kghfh"
Oct 30 21:54:54.470434 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.470325    6133 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mrtmr\" (UniqueName: \"kubernetes.io/projected/209b4e58-e6c2-48f1-8589-f7b6be79720b-kube-api-access-mrtmr\") pod \"dns-default-fqmbc\" (UID: \"209b4e58-e6c2-48f1-8589-f7b6be79720b\") " pod="openshift-dns/dns-default-fqmbc"
Oct 30 21:54:54.470434 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.470362    6133 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/7ddfa67d-20a0-473d-937f-25fd8e0ba80e-node-exporter-wtmp\") pod \"node-exporter-df5jt\" (UID: \"7ddfa67d-20a0-473d-937f-25fd8e0ba80e\") " pod="openshift-monitoring/node-exporter-df5jt"
Oct 30 21:54:54.470434 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.470392    6133 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/446beacf-70a0-4d47-96ea-bbe1ca4a5f28-run-systemd\") pod \"ovnkube-node-mn9vl\" (UID: \"446beacf-70a0-4d47-96ea-bbe1ca4a5f28\") " pod="openshift-ovn-kubernetes/ovnkube-node-mn9vl"
Oct 30 21:54:54.470564 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.470433    6133 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/bd93518f-2f96-4483-9e92-513109dd37cb-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"bd93518f-2f96-4483-9e92-513109dd37cb\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:54:54.470564 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.470474    6133 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/5d5057fc-dd8a-4dce-b8bb-70b2dc3b074c-hostroot\") pod \"multus-jkdxt\" (UID: \"5d5057fc-dd8a-4dce-b8bb-70b2dc3b074c\") " pod="openshift-multus/multus-jkdxt"
Oct 30 21:54:54.470564 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.470507    6133 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/bebdb584-1a96-453d-8a6d-b332e47bec3d-etc-sysconfig\") pod \"tuned-h8csb\" (UID: \"bebdb584-1a96-453d-8a6d-b332e47bec3d\") " pod="openshift-cluster-node-tuning-operator/tuned-h8csb"
Oct 30 21:54:54.470564 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.470533    6133 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r4qz4\" (UniqueName: \"kubernetes.io/projected/bebdb584-1a96-453d-8a6d-b332e47bec3d-kube-api-access-r4qz4\") pod \"tuned-h8csb\" (UID: \"bebdb584-1a96-453d-8a6d-b332e47bec3d\") " pod="openshift-cluster-node-tuning-operator/tuned-h8csb"
Oct 30 21:54:54.470564 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.470552    6133 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/446beacf-70a0-4d47-96ea-bbe1ca4a5f28-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-mn9vl\" (UID: \"446beacf-70a0-4d47-96ea-bbe1ca4a5f28\") " pod="openshift-ovn-kubernetes/ovnkube-node-mn9vl"
Oct 30 21:54:54.470806 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.470605    6133 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/613f514c-6011-4936-8df5-a59b960cd3a6-iptables-alerter-script\") pod \"iptables-alerter-kghfh\" (UID: \"613f514c-6011-4936-8df5-a59b960cd3a6\") " pod="openshift-network-operator/iptables-alerter-kghfh"
Oct 30 21:54:54.470806 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.470613    6133 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/bd93518f-2f96-4483-9e92-513109dd37cb-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"bd93518f-2f96-4483-9e92-513109dd37cb\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:54:54.470806 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.470618    6133 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/bd93518f-2f96-4483-9e92-513109dd37cb-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"bd93518f-2f96-4483-9e92-513109dd37cb\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:54:54.470806 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.470670    6133 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5d5057fc-dd8a-4dce-b8bb-70b2dc3b074c-host-var-lib-kubelet\") pod \"multus-jkdxt\" (UID: \"5d5057fc-dd8a-4dce-b8bb-70b2dc3b074c\") " pod="openshift-multus/multus-jkdxt"
Oct 30 21:54:54.470806 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.470727    6133 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/17da628d-31f5-456c-ad37-52c884d22cf4-metrics-client-ca\") pod \"kube-state-metrics-5dc8d7d89b-8727r\" (UID: \"17da628d-31f5-456c-ad37-52c884d22cf4\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-8727r"
Oct 30 21:54:54.470806 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.470781    6133 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/3e60fceb-8682-45cc-a472-f3d7017c3395-cnibin\") pod \"multus-additional-cni-plugins-c9tnl\" (UID: \"3e60fceb-8682-45cc-a472-f3d7017c3395\") " pod="openshift-multus/multus-additional-cni-plugins-c9tnl"
Oct 30 21:54:54.471227 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.470825    6133 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/613f514c-6011-4936-8df5-a59b960cd3a6-host-slash\") pod \"iptables-alerter-kghfh\" (UID: \"613f514c-6011-4936-8df5-a59b960cd3a6\") " pod="openshift-network-operator/iptables-alerter-kghfh"
Oct 30 21:54:54.471227 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.470849    6133 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6bsvp\" (UniqueName: \"kubernetes.io/projected/3e60fceb-8682-45cc-a472-f3d7017c3395-kube-api-access-6bsvp\") pod \"multus-additional-cni-plugins-c9tnl\" (UID: \"3e60fceb-8682-45cc-a472-f3d7017c3395\") " pod="openshift-multus/multus-additional-cni-plugins-c9tnl"
Oct 30 21:54:54.471227 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.470895    6133 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5d5057fc-dd8a-4dce-b8bb-70b2dc3b074c-cni-binary-copy\") pod \"multus-jkdxt\" (UID: \"5d5057fc-dd8a-4dce-b8bb-70b2dc3b074c\") " pod="openshift-multus/multus-jkdxt"
Oct 30 21:54:54.471227 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.471018    6133 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/446beacf-70a0-4d47-96ea-bbe1ca4a5f28-host-run-netns\") pod \"ovnkube-node-mn9vl\" (UID: \"446beacf-70a0-4d47-96ea-bbe1ca4a5f28\") " pod="openshift-ovn-kubernetes/ovnkube-node-mn9vl"
Oct 30 21:54:54.471227 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.471067    6133 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/446beacf-70a0-4d47-96ea-bbe1ca4a5f28-run-openvswitch\") pod \"ovnkube-node-mn9vl\" (UID: \"446beacf-70a0-4d47-96ea-bbe1ca4a5f28\") " pod="openshift-ovn-kubernetes/ovnkube-node-mn9vl"
Oct 30 21:54:54.471227 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.471117    6133 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/446beacf-70a0-4d47-96ea-bbe1ca4a5f28-env-overrides\") pod \"ovnkube-node-mn9vl\" (UID: \"446beacf-70a0-4d47-96ea-bbe1ca4a5f28\") " pod="openshift-ovn-kubernetes/ovnkube-node-mn9vl"
Oct 30 21:54:54.471227 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.471143    6133 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5d5057fc-dd8a-4dce-b8bb-70b2dc3b074c-cni-binary-copy\") pod \"multus-jkdxt\" (UID: \"5d5057fc-dd8a-4dce-b8bb-70b2dc3b074c\") " pod="openshift-multus/multus-jkdxt"
Oct 30 21:54:54.471227 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.471159    6133 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bd93518f-2f96-4483-9e92-513109dd37cb-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"bd93518f-2f96-4483-9e92-513109dd37cb\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:54:54.471227 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.471191    6133 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ljsqq\" (UniqueName: \"kubernetes.io/projected/32ae0d42-4b76-47b7-b9d6-26b09f00ba5d-kube-api-access-ljsqq\") pod \"network-check-target-fvbz8\" (UID: \"32ae0d42-4b76-47b7-b9d6-26b09f00ba5d\") " pod="openshift-network-diagnostics/network-check-target-fvbz8"
Oct 30 21:54:54.471586 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.471231    6133 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/6192d2be-bba7-4099-b54c-e7e9b8d75389-data-volume\") pod \"insights-runtime-extractor-6jppj\" (UID: \"6192d2be-bba7-4099-b54c-e7e9b8d75389\") " pod="openshift-insights/insights-runtime-extractor-6jppj"
Oct 30 21:54:54.471586 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.471316    6133 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5d5057fc-dd8a-4dce-b8bb-70b2dc3b074c-etc-kubernetes\") pod \"multus-jkdxt\" (UID: \"5d5057fc-dd8a-4dce-b8bb-70b2dc3b074c\") " pod="openshift-multus/multus-jkdxt"
Oct 30 21:54:54.471586 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.471342    6133 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/7ddfa67d-20a0-473d-937f-25fd8e0ba80e-root\") pod \"node-exporter-df5jt\" (UID: \"7ddfa67d-20a0-473d-937f-25fd8e0ba80e\") " pod="openshift-monitoring/node-exporter-df5jt"
Oct 30 21:54:54.471586 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.471343    6133 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/446beacf-70a0-4d47-96ea-bbe1ca4a5f28-env-overrides\") pod \"ovnkube-node-mn9vl\" (UID: \"446beacf-70a0-4d47-96ea-bbe1ca4a5f28\") " pod="openshift-ovn-kubernetes/ovnkube-node-mn9vl"
Oct 30 21:54:54.471586 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.471386    6133 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/17da628d-31f5-456c-ad37-52c884d22cf4-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-5dc8d7d89b-8727r\" (UID: \"17da628d-31f5-456c-ad37-52c884d22cf4\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-8727r"
Oct 30 21:54:54.471586 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.471443    6133 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/6192d2be-bba7-4099-b54c-e7e9b8d75389-data-volume\") pod \"insights-runtime-extractor-6jppj\" (UID: \"6192d2be-bba7-4099-b54c-e7e9b8d75389\") " pod="openshift-insights/insights-runtime-extractor-6jppj"
Oct 30 21:54:54.471894 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.471420    6133 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/15e1de31-92a1-4179-8ec1-400346880d1d-etc-selinux\") pod \"aws-ebs-csi-driver-node-kqbp9\" (UID: \"15e1de31-92a1-4179-8ec1-400346880d1d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kqbp9"
Oct 30 21:54:54.471894 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.471465    6133 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/17da628d-31f5-456c-ad37-52c884d22cf4-metrics-client-ca\") pod \"kube-state-metrics-5dc8d7d89b-8727r\" (UID: \"17da628d-31f5-456c-ad37-52c884d22cf4\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-8727r"
Oct 30 21:54:54.471894 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.471416    6133 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/bd93518f-2f96-4483-9e92-513109dd37cb-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"bd93518f-2f96-4483-9e92-513109dd37cb\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:54:54.471894 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.471809    6133 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bd93518f-2f96-4483-9e92-513109dd37cb-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"bd93518f-2f96-4483-9e92-513109dd37cb\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:54:54.471894 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.471882    6133 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/17da628d-31f5-456c-ad37-52c884d22cf4-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-5dc8d7d89b-8727r\" (UID: \"17da628d-31f5-456c-ad37-52c884d22cf4\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-8727r"
Oct 30 21:54:54.472136 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.471899    6133 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/446beacf-70a0-4d47-96ea-bbe1ca4a5f28-run-ovn\") pod \"ovnkube-node-mn9vl\" (UID: \"446beacf-70a0-4d47-96ea-bbe1ca4a5f28\") " pod="openshift-ovn-kubernetes/ovnkube-node-mn9vl"
Oct 30 21:54:54.475244 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.475219    6133 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/6192d2be-bba7-4099-b54c-e7e9b8d75389-crio-socket\") pod \"insights-runtime-extractor-6jppj\" (UID: \"6192d2be-bba7-4099-b54c-e7e9b8d75389\") " pod="openshift-insights/insights-runtime-extractor-6jppj"
Oct 30 21:54:54.475356 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.475313    6133 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/bd93518f-2f96-4483-9e92-513109dd37cb-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"bd93518f-2f96-4483-9e92-513109dd37cb\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:54:54.475664 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.475645    6133 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/3e60fceb-8682-45cc-a472-f3d7017c3395-tuning-conf-dir\") pod \"multus-additional-cni-plugins-c9tnl\" (UID: \"3e60fceb-8682-45cc-a472-f3d7017c3395\") " pod="openshift-multus/multus-additional-cni-plugins-c9tnl"
Oct 30 21:54:54.475715 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.475686    6133 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/3e60fceb-8682-45cc-a472-f3d7017c3395-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-c9tnl\" (UID: \"3e60fceb-8682-45cc-a472-f3d7017c3395\") " pod="openshift-multus/multus-additional-cni-plugins-c9tnl"
Oct 30 21:54:54.475748 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.475716    6133 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4621048c-230e-441f-85fc-423e4ba525ec-cert\") pod \"ingress-canary-fl5mx\" (UID: \"4621048c-230e-441f-85fc-423e4ba525ec\") " pod="openshift-ingress-canary/ingress-canary-fl5mx"
Oct 30 21:54:54.475791 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.475745    6133 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wsnr5\" (UniqueName: \"kubernetes.io/projected/7ddfa67d-20a0-473d-937f-25fd8e0ba80e-kube-api-access-wsnr5\") pod \"node-exporter-df5jt\" (UID: \"7ddfa67d-20a0-473d-937f-25fd8e0ba80e\") " pod="openshift-monitoring/node-exporter-df5jt"
Oct 30 21:54:54.475838 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.475818    6133 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/532658b6-c86e-4feb-89cb-14a91316a7a0-metrics-certs\") pod \"network-metrics-daemon-n8xrh\" (UID: \"532658b6-c86e-4feb-89cb-14a91316a7a0\") " pod="openshift-multus/network-metrics-daemon-n8xrh"
Oct 30 21:54:54.475884 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.475843    6133 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7ddfa67d-20a0-473d-937f-25fd8e0ba80e-metrics-client-ca\") pod \"node-exporter-df5jt\" (UID: \"7ddfa67d-20a0-473d-937f-25fd8e0ba80e\") " pod="openshift-monitoring/node-exporter-df5jt"
Oct 30 21:54:54.475884 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.475869    6133 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/7ddfa67d-20a0-473d-937f-25fd8e0ba80e-node-exporter-accelerators-collector-config\") pod \"node-exporter-df5jt\" (UID: \"7ddfa67d-20a0-473d-937f-25fd8e0ba80e\") " pod="openshift-monitoring/node-exporter-df5jt"
Oct 30 21:54:54.476090 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.475897    6133 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/15e1de31-92a1-4179-8ec1-400346880d1d-kubelet-dir\") pod \"aws-ebs-csi-driver-node-kqbp9\" (UID: \"15e1de31-92a1-4179-8ec1-400346880d1d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kqbp9"
Oct 30 21:54:54.476090 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.475906    6133 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/3e60fceb-8682-45cc-a472-f3d7017c3395-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-c9tnl\" (UID: \"3e60fceb-8682-45cc-a472-f3d7017c3395\") " pod="openshift-multus/multus-additional-cni-plugins-c9tnl"
Oct 30 21:54:54.476090 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.475923    6133 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/bebdb584-1a96-453d-8a6d-b332e47bec3d-etc-sysctl-d\") pod \"tuned-h8csb\" (UID: \"bebdb584-1a96-453d-8a6d-b332e47bec3d\") " pod="openshift-cluster-node-tuning-operator/tuned-h8csb"
Oct 30 21:54:54.476090 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.475947    6133 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5d5057fc-dd8a-4dce-b8bb-70b2dc3b074c-host-var-lib-cni-bin\") pod \"multus-jkdxt\" (UID: \"5d5057fc-dd8a-4dce-b8bb-70b2dc3b074c\") " pod="openshift-multus/multus-jkdxt"
Oct 30 21:54:54.476090 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.475973    6133 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/fbd8f09b-9962-4910-8b25-267cbbb3ea5d-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-868d6f78df-nx2k6\" (UID: \"fbd8f09b-9962-4910-8b25-267cbbb3ea5d\") " pod="openshift-monitoring/thanos-querier-868d6f78df-nx2k6"
Oct 30 21:54:54.476090 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.476022    6133 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w78hr\" (UniqueName: \"kubernetes.io/projected/9d2cced5-9ceb-421f-b831-8370a5ee0ca9-kube-api-access-w78hr\") pod \"monitoring-plugin-77f4f76594-htsvq\" (UID: \"9d2cced5-9ceb-421f-b831-8370a5ee0ca9\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-htsvq"
Oct 30 21:54:54.476090 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.476048    6133 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qrt69\" (UniqueName: \"kubernetes.io/projected/bd93518f-2f96-4483-9e92-513109dd37cb-kube-api-access-qrt69\") pod \"prometheus-k8s-0\" (UID: \"bd93518f-2f96-4483-9e92-513109dd37cb\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:54:54.476090 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.476063    6133 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4621048c-230e-441f-85fc-423e4ba525ec-cert\") pod \"ingress-canary-fl5mx\" (UID: \"4621048c-230e-441f-85fc-423e4ba525ec\") " pod="openshift-ingress-canary/ingress-canary-fl5mx"
Oct 30 21:54:54.476090 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.476070    6133 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5d5057fc-dd8a-4dce-b8bb-70b2dc3b074c-system-cni-dir\") pod \"multus-jkdxt\" (UID: \"5d5057fc-dd8a-4dce-b8bb-70b2dc3b074c\") " pod="openshift-multus/multus-jkdxt"
Oct 30 21:54:54.476501 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.476122    6133 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/7ddfa67d-20a0-473d-937f-25fd8e0ba80e-node-exporter-accelerators-collector-config\") pod \"node-exporter-df5jt\" (UID: \"7ddfa67d-20a0-473d-937f-25fd8e0ba80e\") " pod="openshift-monitoring/node-exporter-df5jt"
Oct 30 21:54:54.476501 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.476129    6133 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/5d5057fc-dd8a-4dce-b8bb-70b2dc3b074c-host-run-k8s-cni-cncf-io\") pod \"multus-jkdxt\" (UID: \"5d5057fc-dd8a-4dce-b8bb-70b2dc3b074c\") " pod="openshift-multus/multus-jkdxt"
Oct 30 21:54:54.476501 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.476122    6133 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/532658b6-c86e-4feb-89cb-14a91316a7a0-metrics-certs\") pod \"network-metrics-daemon-n8xrh\" (UID: \"532658b6-c86e-4feb-89cb-14a91316a7a0\") " pod="openshift-multus/network-metrics-daemon-n8xrh"
Oct 30 21:54:54.476501 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.476229    6133 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7ddfa67d-20a0-473d-937f-25fd8e0ba80e-metrics-client-ca\") pod \"node-exporter-df5jt\" (UID: \"7ddfa67d-20a0-473d-937f-25fd8e0ba80e\") " pod="openshift-monitoring/node-exporter-df5jt"
Oct 30 21:54:54.476501 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.476356    6133 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bebdb584-1a96-453d-8a6d-b332e47bec3d-etc-kubernetes\") pod \"tuned-h8csb\" (UID: \"bebdb584-1a96-453d-8a6d-b332e47bec3d\") " pod="openshift-cluster-node-tuning-operator/tuned-h8csb"
Oct 30 21:54:54.476501 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.476386    6133 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/15e1de31-92a1-4179-8ec1-400346880d1d-device-dir\") pod \"aws-ebs-csi-driver-node-kqbp9\" (UID: \"15e1de31-92a1-4179-8ec1-400346880d1d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kqbp9"
Oct 30 21:54:54.476501 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.476430    6133 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/2bcee03a-9b40-4155-8d81-43dece439162-agent-certs\") pod \"konnectivity-agent-thksb\" (UID: \"2bcee03a-9b40-4155-8d81-43dece439162\") " pod="kube-system/konnectivity-agent-thksb"
Oct 30 21:54:54.476501 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.476483    6133 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/fbd8f09b-9962-4910-8b25-267cbbb3ea5d-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-868d6f78df-nx2k6\" (UID: \"fbd8f09b-9962-4910-8b25-267cbbb3ea5d\") " pod="openshift-monitoring/thanos-querier-868d6f78df-nx2k6"
Oct 30 21:54:54.476501 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.476489    6133 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/209b4e58-e6c2-48f1-8589-f7b6be79720b-metrics-tls\") pod \"dns-default-fqmbc\" (UID: \"209b4e58-e6c2-48f1-8589-f7b6be79720b\") " pod="openshift-dns/dns-default-fqmbc"
Oct 30 21:54:54.476942 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.476538    6133 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3e60fceb-8682-45cc-a472-f3d7017c3395-system-cni-dir\") pod \"multus-additional-cni-plugins-c9tnl\" (UID: \"3e60fceb-8682-45cc-a472-f3d7017c3395\") " pod="openshift-multus/multus-additional-cni-plugins-c9tnl"
Oct 30 21:54:54.476942 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.476595    6133 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5d5057fc-dd8a-4dce-b8bb-70b2dc3b074c-cnibin\") pod \"multus-jkdxt\" (UID: \"5d5057fc-dd8a-4dce-b8bb-70b2dc3b074c\") " pod="openshift-multus/multus-jkdxt"
Oct 30 21:54:54.476942 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.476625    6133 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kncjh\" (UniqueName: \"kubernetes.io/projected/5d5057fc-dd8a-4dce-b8bb-70b2dc3b074c-kube-api-access-kncjh\") pod \"multus-jkdxt\" (UID: \"5d5057fc-dd8a-4dce-b8bb-70b2dc3b074c\") " pod="openshift-multus/multus-jkdxt"
Oct 30 21:54:54.476942 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.476654    6133 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7ddfa67d-20a0-473d-937f-25fd8e0ba80e-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-df5jt\" (UID: \"7ddfa67d-20a0-473d-937f-25fd8e0ba80e\") " pod="openshift-monitoring/node-exporter-df5jt"
Oct 30 21:54:54.476942 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.476656    6133 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/2bcee03a-9b40-4155-8d81-43dece439162-agent-certs\") pod \"konnectivity-agent-thksb\" (UID: \"2bcee03a-9b40-4155-8d81-43dece439162\") " pod="kube-system/konnectivity-agent-thksb"
Oct 30 21:54:54.476942 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.476679    6133 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/446beacf-70a0-4d47-96ea-bbe1ca4a5f28-systemd-units\") pod \"ovnkube-node-mn9vl\" (UID: \"446beacf-70a0-4d47-96ea-bbe1ca4a5f28\") " pod="openshift-ovn-kubernetes/ovnkube-node-mn9vl"
Oct 30 21:54:54.476942 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.476705    6133 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/bd93518f-2f96-4483-9e92-513109dd37cb-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"bd93518f-2f96-4483-9e92-513109dd37cb\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:54:54.476942 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.476729    6133 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/5d5057fc-dd8a-4dce-b8bb-70b2dc3b074c-multus-daemon-config\") pod \"multus-jkdxt\" (UID: \"5d5057fc-dd8a-4dce-b8bb-70b2dc3b074c\") " pod="openshift-multus/multus-jkdxt"
Oct 30 21:54:54.476942 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.476765    6133 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/af83b188-09cb-4621-a9da-be33856d19f0-original-pull-secret\") pod \"global-pull-secret-syncer-q7fwz\" (UID: \"af83b188-09cb-4621-a9da-be33856d19f0\") " pod="kube-system/global-pull-secret-syncer-q7fwz"
Oct 30 21:54:54.476942 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.476776    6133 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/209b4e58-e6c2-48f1-8589-f7b6be79720b-metrics-tls\") pod \"dns-default-fqmbc\" (UID: \"209b4e58-e6c2-48f1-8589-f7b6be79720b\") " pod="openshift-dns/dns-default-fqmbc"
Oct 30 21:54:54.476942 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.476798    6133 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/bd93518f-2f96-4483-9e92-513109dd37cb-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"bd93518f-2f96-4483-9e92-513109dd37cb\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:54:54.476942 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.476818    6133 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7ddfa67d-20a0-473d-937f-25fd8e0ba80e-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-df5jt\" (UID: \"7ddfa67d-20a0-473d-937f-25fd8e0ba80e\") " pod="openshift-monitoring/node-exporter-df5jt"
Oct 30 21:54:54.476942 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.476825    6133 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5d5057fc-dd8a-4dce-b8bb-70b2dc3b074c-multus-conf-dir\") pod \"multus-jkdxt\" (UID: \"5d5057fc-dd8a-4dce-b8bb-70b2dc3b074c\") " pod="openshift-multus/multus-jkdxt"
Oct 30 21:54:54.476942 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.476851    6133 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/fe472529-1294-46a1-a666-dc00d5d37eca-serviceca\") pod \"node-ca-qzgkj\" (UID: \"fe472529-1294-46a1-a666-dc00d5d37eca\") " pod="openshift-image-registry/node-ca-qzgkj"
Oct 30 21:54:54.476942 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.476878    6133 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/bebdb584-1a96-453d-8a6d-b332e47bec3d-etc-sysctl-conf\") pod \"tuned-h8csb\" (UID: \"bebdb584-1a96-453d-8a6d-b332e47bec3d\") " pod="openshift-cluster-node-tuning-operator/tuned-h8csb"
Oct 30 21:54:54.476942 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.476883    6133 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/bd93518f-2f96-4483-9e92-513109dd37cb-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"bd93518f-2f96-4483-9e92-513109dd37cb\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:54:54.476942 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.476901    6133 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/2bcee03a-9b40-4155-8d81-43dece439162-konnectivity-ca\") pod \"konnectivity-agent-thksb\" (UID: \"2bcee03a-9b40-4155-8d81-43dece439162\") " pod="kube-system/konnectivity-agent-thksb"
Oct 30 21:54:54.476942 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.476944    6133 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/446beacf-70a0-4d47-96ea-bbe1ca4a5f28-host-run-ovn-kubernetes\") pod \"ovnkube-node-mn9vl\" (UID: \"446beacf-70a0-4d47-96ea-bbe1ca4a5f28\") " pod="openshift-ovn-kubernetes/ovnkube-node-mn9vl"
Oct 30 21:54:54.477722 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.476962    6133 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/5d5057fc-dd8a-4dce-b8bb-70b2dc3b074c-multus-daemon-config\") pod \"multus-jkdxt\" (UID: \"5d5057fc-dd8a-4dce-b8bb-70b2dc3b074c\") " pod="openshift-multus/multus-jkdxt"
Oct 30 21:54:54.477722 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.476980    6133 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/446beacf-70a0-4d47-96ea-bbe1ca4a5f28-ovnkube-config\") pod \"ovnkube-node-mn9vl\" (UID: \"446beacf-70a0-4d47-96ea-bbe1ca4a5f28\") " pod="openshift-ovn-kubernetes/ovnkube-node-mn9vl"
Oct 30 21:54:54.477722 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.477044    6133 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jw4wk\" (UniqueName: \"kubernetes.io/projected/6192d2be-bba7-4099-b54c-e7e9b8d75389-kube-api-access-jw4wk\") pod \"insights-runtime-extractor-6jppj\" (UID: \"6192d2be-bba7-4099-b54c-e7e9b8d75389\") " pod="openshift-insights/insights-runtime-extractor-6jppj"
Oct 30 21:54:54.477722 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.477081    6133 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/bd93518f-2f96-4483-9e92-513109dd37cb-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"bd93518f-2f96-4483-9e92-513109dd37cb\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:54:54.477722 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.477106    6133 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/bd93518f-2f96-4483-9e92-513109dd37cb-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"bd93518f-2f96-4483-9e92-513109dd37cb\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:54:54.477722 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.477144    6133 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/af83b188-09cb-4621-a9da-be33856d19f0-original-pull-secret\") pod \"global-pull-secret-syncer-q7fwz\" (UID: \"af83b188-09cb-4621-a9da-be33856d19f0\") " pod="kube-system/global-pull-secret-syncer-q7fwz"
Oct 30 21:54:54.477722 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.477228    6133 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r6wtm\" (UniqueName: \"kubernetes.io/projected/532658b6-c86e-4feb-89cb-14a91316a7a0-kube-api-access-r6wtm\") pod \"network-metrics-daemon-n8xrh\" (UID: \"532658b6-c86e-4feb-89cb-14a91316a7a0\") " pod="openshift-multus/network-metrics-daemon-n8xrh"
Oct 30 21:54:54.477722 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.477241    6133 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/2bcee03a-9b40-4155-8d81-43dece439162-konnectivity-ca\") pod \"konnectivity-agent-thksb\" (UID: \"2bcee03a-9b40-4155-8d81-43dece439162\") " pod="kube-system/konnectivity-agent-thksb"
Oct 30 21:54:54.477722 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.477243    6133 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/446beacf-70a0-4d47-96ea-bbe1ca4a5f28-ovnkube-config\") pod \"ovnkube-node-mn9vl\" (UID: \"446beacf-70a0-4d47-96ea-bbe1ca4a5f28\") " pod="openshift-ovn-kubernetes/ovnkube-node-mn9vl"
Oct 30 21:54:54.477722 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.477338    6133 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6192d2be-bba7-4099-b54c-e7e9b8d75389-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6jppj\" (UID: \"6192d2be-bba7-4099-b54c-e7e9b8d75389\") " pod="openshift-insights/insights-runtime-extractor-6jppj"
Oct 30 21:54:54.477722 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.477366    6133 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/bd93518f-2f96-4483-9e92-513109dd37cb-config\") pod \"prometheus-k8s-0\" (UID: \"bd93518f-2f96-4483-9e92-513109dd37cb\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:54:54.477722 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.477393    6133 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/bebdb584-1a96-453d-8a6d-b332e47bec3d-var-lib-kubelet\") pod \"tuned-h8csb\" (UID: \"bebdb584-1a96-453d-8a6d-b332e47bec3d\") " pod="openshift-cluster-node-tuning-operator/tuned-h8csb"
Oct 30 21:54:54.477722 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.477420    6133 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/484e37a0-fd1d-4fd0-a200-939b8945984b-metrics-client-ca\") pod \"openshift-state-metrics-7bd7489957-f26w7\" (UID: \"484e37a0-fd1d-4fd0-a200-939b8945984b\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-f26w7"
Oct 30 21:54:54.477722 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.477451    6133 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/bd93518f-2f96-4483-9e92-513109dd37cb-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"bd93518f-2f96-4483-9e92-513109dd37cb\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:54:54.477722 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.477445    6133 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5d5057fc-dd8a-4dce-b8bb-70b2dc3b074c-multus-cni-dir\") pod \"multus-jkdxt\" (UID: \"5d5057fc-dd8a-4dce-b8bb-70b2dc3b074c\") " pod="openshift-multus/multus-jkdxt"
Oct 30 21:54:54.477722 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.477477    6133 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/fe472529-1294-46a1-a666-dc00d5d37eca-serviceca\") pod \"node-ca-qzgkj\" (UID: \"fe472529-1294-46a1-a666-dc00d5d37eca\") " pod="openshift-image-registry/node-ca-qzgkj"
Oct 30 21:54:54.477722 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.477515    6133 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/9d2cced5-9ceb-421f-b831-8370a5ee0ca9-monitoring-plugin-cert\") pod \"monitoring-plugin-77f4f76594-htsvq\" (UID: \"9d2cced5-9ceb-421f-b831-8370a5ee0ca9\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-htsvq"
Oct 30 21:54:54.477722 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.477579    6133 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/15e1de31-92a1-4179-8ec1-400346880d1d-registration-dir\") pod \"aws-ebs-csi-driver-node-kqbp9\" (UID: \"15e1de31-92a1-4179-8ec1-400346880d1d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kqbp9"
Oct 30 21:54:54.477722 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.477579    6133 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/bd93518f-2f96-4483-9e92-513109dd37cb-config\") pod \"prometheus-k8s-0\" (UID: \"bd93518f-2f96-4483-9e92-513109dd37cb\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:54:54.477722 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.477596    6133 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6192d2be-bba7-4099-b54c-e7e9b8d75389-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6jppj\" (UID: \"6192d2be-bba7-4099-b54c-e7e9b8d75389\") " pod="openshift-insights/insights-runtime-extractor-6jppj"
Oct 30 21:54:54.477722 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.477647    6133 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/15e1de31-92a1-4179-8ec1-400346880d1d-sys-fs\") pod \"aws-ebs-csi-driver-node-kqbp9\" (UID: \"15e1de31-92a1-4179-8ec1-400346880d1d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kqbp9"
Oct 30 21:54:54.477722 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.477672    6133 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/484e37a0-fd1d-4fd0-a200-939b8945984b-metrics-client-ca\") pod \"openshift-state-metrics-7bd7489957-f26w7\" (UID: \"484e37a0-fd1d-4fd0-a200-939b8945984b\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-f26w7"
Oct 30 21:54:54.477722 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.477678    6133 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bebdb584-1a96-453d-8a6d-b332e47bec3d-tmp\") pod \"tuned-h8csb\" (UID: \"bebdb584-1a96-453d-8a6d-b332e47bec3d\") " pod="openshift-cluster-node-tuning-operator/tuned-h8csb"
Oct 30 21:54:54.477722 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.477718    6133 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h6b6r\" (UniqueName: \"kubernetes.io/projected/613f514c-6011-4936-8df5-a59b960cd3a6-kube-api-access-h6b6r\") pod \"iptables-alerter-kghfh\" (UID: \"613f514c-6011-4936-8df5-a59b960cd3a6\") " pod="openshift-network-operator/iptables-alerter-kghfh"
Oct 30 21:54:54.478647 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.477760    6133 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/9d2cced5-9ceb-421f-b831-8370a5ee0ca9-monitoring-plugin-cert\") pod \"monitoring-plugin-77f4f76594-htsvq\" (UID: \"9d2cced5-9ceb-421f-b831-8370a5ee0ca9\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-htsvq"
Oct 30 21:54:54.478647 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.477770    6133 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bebdb584-1a96-453d-8a6d-b332e47bec3d-tmp\") pod \"tuned-h8csb\" (UID: \"bebdb584-1a96-453d-8a6d-b332e47bec3d\") " pod="openshift-cluster-node-tuning-operator/tuned-h8csb"
Oct 30 21:54:54.478647 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.477760    6133 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/209b4e58-e6c2-48f1-8589-f7b6be79720b-config-volume\") pod \"dns-default-fqmbc\" (UID: \"209b4e58-e6c2-48f1-8589-f7b6be79720b\") " pod="openshift-dns/dns-default-fqmbc"
Oct 30 21:54:54.478647 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.477807    6133 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/bd93518f-2f96-4483-9e92-513109dd37cb-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"bd93518f-2f96-4483-9e92-513109dd37cb\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:54:54.478647 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.477834    6133 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lgk4w\" (UniqueName: \"kubernetes.io/projected/15e1de31-92a1-4179-8ec1-400346880d1d-kube-api-access-lgk4w\") pod \"aws-ebs-csi-driver-node-kqbp9\" (UID: \"15e1de31-92a1-4179-8ec1-400346880d1d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kqbp9"
Oct 30 21:54:54.478647 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.477859    6133 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/bebdb584-1a96-453d-8a6d-b332e47bec3d-run\") pod \"tuned-h8csb\" (UID: \"bebdb584-1a96-453d-8a6d-b332e47bec3d\") " pod="openshift-cluster-node-tuning-operator/tuned-h8csb"
Oct 30 21:54:54.478647 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.477887    6133 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/484e37a0-fd1d-4fd0-a200-939b8945984b-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-7bd7489957-f26w7\" (UID: \"484e37a0-fd1d-4fd0-a200-939b8945984b\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-f26w7"
Oct 30 21:54:54.478647 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.477927    6133 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/bd93518f-2f96-4483-9e92-513109dd37cb-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"bd93518f-2f96-4483-9e92-513109dd37cb\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:54:54.478647 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.477947    6133 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/209b4e58-e6c2-48f1-8589-f7b6be79720b-config-volume\") pod \"dns-default-fqmbc\" (UID: \"209b4e58-e6c2-48f1-8589-f7b6be79720b\") " pod="openshift-dns/dns-default-fqmbc"
Oct 30 21:54:54.478647 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.477967    6133 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zzw6w\" (UniqueName: \"kubernetes.io/projected/fbd8f09b-9962-4910-8b25-267cbbb3ea5d-kube-api-access-zzw6w\") pod \"thanos-querier-868d6f78df-nx2k6\" (UID: \"fbd8f09b-9962-4910-8b25-267cbbb3ea5d\") " pod="openshift-monitoring/thanos-querier-868d6f78df-nx2k6"
Oct 30 21:54:54.478647 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.478013    6133 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/446beacf-70a0-4d47-96ea-bbe1ca4a5f28-host-slash\") pod \"ovnkube-node-mn9vl\" (UID: \"446beacf-70a0-4d47-96ea-bbe1ca4a5f28\") " pod="openshift-ovn-kubernetes/ovnkube-node-mn9vl"
Oct 30 21:54:54.478647 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.478091    6133 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/484e37a0-fd1d-4fd0-a200-939b8945984b-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-7bd7489957-f26w7\" (UID: \"484e37a0-fd1d-4fd0-a200-939b8945984b\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-f26w7"
Oct 30 21:54:54.478647 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.478117    6133 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/446beacf-70a0-4d47-96ea-bbe1ca4a5f28-node-log\") pod \"ovnkube-node-mn9vl\" (UID: \"446beacf-70a0-4d47-96ea-bbe1ca4a5f28\") " pod="openshift-ovn-kubernetes/ovnkube-node-mn9vl"
Oct 30 21:54:54.478647 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.478156    6133 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/bd93518f-2f96-4483-9e92-513109dd37cb-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"bd93518f-2f96-4483-9e92-513109dd37cb\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:54:54.478647 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.478214    6133 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/5d5057fc-dd8a-4dce-b8bb-70b2dc3b074c-host-run-multus-certs\") pod \"multus-jkdxt\" (UID: \"5d5057fc-dd8a-4dce-b8bb-70b2dc3b074c\") " pod="openshift-multus/multus-jkdxt"
Oct 30 21:54:54.478647 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.478252    6133 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/bebdb584-1a96-453d-8a6d-b332e47bec3d-etc-modprobe-d\") pod \"tuned-h8csb\" (UID: \"bebdb584-1a96-453d-8a6d-b332e47bec3d\") " pod="openshift-cluster-node-tuning-operator/tuned-h8csb"
Oct 30 21:54:54.478647 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.478280    6133 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/7ddfa67d-20a0-473d-937f-25fd8e0ba80e-node-exporter-textfile\") pod \"node-exporter-df5jt\" (UID: \"7ddfa67d-20a0-473d-937f-25fd8e0ba80e\") " pod="openshift-monitoring/node-exporter-df5jt"
Oct 30 21:54:54.478647 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.478313    6133 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/7ddfa67d-20a0-473d-937f-25fd8e0ba80e-node-exporter-tls\") pod \"node-exporter-df5jt\" (UID: \"7ddfa67d-20a0-473d-937f-25fd8e0ba80e\") " pod="openshift-monitoring/node-exporter-df5jt"
Oct 30 21:54:54.478647 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.478319    6133 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/bd93518f-2f96-4483-9e92-513109dd37cb-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"bd93518f-2f96-4483-9e92-513109dd37cb\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:54:54.478647 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.478357    6133 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/fe472529-1294-46a1-a666-dc00d5d37eca-host\") pod \"node-ca-qzgkj\" (UID: \"fe472529-1294-46a1-a666-dc00d5d37eca\") " pod="openshift-image-registry/node-ca-qzgkj"
Oct 30 21:54:54.478647 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.478386    6133 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/446beacf-70a0-4d47-96ea-bbe1ca4a5f28-host-cni-bin\") pod \"ovnkube-node-mn9vl\" (UID: \"446beacf-70a0-4d47-96ea-bbe1ca4a5f28\") " pod="openshift-ovn-kubernetes/ovnkube-node-mn9vl"
Oct 30 21:54:54.478647 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.478386    6133 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/7ddfa67d-20a0-473d-937f-25fd8e0ba80e-node-exporter-textfile\") pod \"node-exporter-df5jt\" (UID: \"7ddfa67d-20a0-473d-937f-25fd8e0ba80e\") " pod="openshift-monitoring/node-exporter-df5jt"
Oct 30 21:54:54.478647 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.478440    6133 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mkrsw\" (UniqueName: \"kubernetes.io/projected/446beacf-70a0-4d47-96ea-bbe1ca4a5f28-kube-api-access-mkrsw\") pod \"ovnkube-node-mn9vl\" (UID: \"446beacf-70a0-4d47-96ea-bbe1ca4a5f28\") " pod="openshift-ovn-kubernetes/ovnkube-node-mn9vl"
Oct 30 21:54:54.478647 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.478520    6133 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/7ddfa67d-20a0-473d-937f-25fd8e0ba80e-node-exporter-tls\") pod \"node-exporter-df5jt\" (UID: \"7ddfa67d-20a0-473d-937f-25fd8e0ba80e\") " pod="openshift-monitoring/node-exporter-df5jt"
Oct 30 21:54:54.478647 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.478597    6133 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/bd93518f-2f96-4483-9e92-513109dd37cb-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"bd93518f-2f96-4483-9e92-513109dd37cb\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:54:54.478647 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.478639    6133 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/bd93518f-2f96-4483-9e92-513109dd37cb-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"bd93518f-2f96-4483-9e92-513109dd37cb\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:54:54.478647 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.478666    6133 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/5d5057fc-dd8a-4dce-b8bb-70b2dc3b074c-multus-socket-dir-parent\") pod \"multus-jkdxt\" (UID: \"5d5057fc-dd8a-4dce-b8bb-70b2dc3b074c\") " pod="openshift-multus/multus-jkdxt"
Oct 30 21:54:54.479831 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.478705    6133 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/484e37a0-fd1d-4fd0-a200-939b8945984b-openshift-state-metrics-tls\") pod \"openshift-state-metrics-7bd7489957-f26w7\" (UID: \"484e37a0-fd1d-4fd0-a200-939b8945984b\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-f26w7"
Oct 30 21:54:54.479831 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.478745    6133 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/446beacf-70a0-4d47-96ea-bbe1ca4a5f28-etc-openvswitch\") pod \"ovnkube-node-mn9vl\" (UID: \"446beacf-70a0-4d47-96ea-bbe1ca4a5f28\") " pod="openshift-ovn-kubernetes/ovnkube-node-mn9vl"
Oct 30 21:54:54.479831 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.478876    6133 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/bd93518f-2f96-4483-9e92-513109dd37cb-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"bd93518f-2f96-4483-9e92-513109dd37cb\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:54:54.479831 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.478904    6133 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/446beacf-70a0-4d47-96ea-bbe1ca4a5f28-ovnkube-script-lib\") pod \"ovnkube-node-mn9vl\" (UID: \"446beacf-70a0-4d47-96ea-bbe1ca4a5f28\") " pod="openshift-ovn-kubernetes/ovnkube-node-mn9vl"
Oct 30 21:54:54.479831 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.478945    6133 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/bd93518f-2f96-4483-9e92-513109dd37cb-config-out\") pod \"prometheus-k8s-0\" (UID: \"bd93518f-2f96-4483-9e92-513109dd37cb\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:54:54.479831 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.478972    6133 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/bd93518f-2f96-4483-9e92-513109dd37cb-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"bd93518f-2f96-4483-9e92-513109dd37cb\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:54:54.479831 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.478907    6133 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/bd93518f-2f96-4483-9e92-513109dd37cb-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"bd93518f-2f96-4483-9e92-513109dd37cb\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:54:54.479831 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.479013    6133 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jp25c\" (UniqueName: \"kubernetes.io/projected/fc4a67d9-8c7c-4f71-9a8a-2bba09329bc6-kube-api-access-jp25c\") pod \"node-resolver-wwb9f\" (UID: \"fc4a67d9-8c7c-4f71-9a8a-2bba09329bc6\") " pod="openshift-dns/node-resolver-wwb9f"
Oct 30 21:54:54.479831 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.479044    6133 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/15e1de31-92a1-4179-8ec1-400346880d1d-socket-dir\") pod \"aws-ebs-csi-driver-node-kqbp9\" (UID: \"15e1de31-92a1-4179-8ec1-400346880d1d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kqbp9"
Oct 30 21:54:54.479831 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.479061    6133 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/bd93518f-2f96-4483-9e92-513109dd37cb-config-out\") pod \"prometheus-k8s-0\" (UID: \"bd93518f-2f96-4483-9e92-513109dd37cb\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:54:54.479831 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.479073    6133 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/bd93518f-2f96-4483-9e92-513109dd37cb-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"bd93518f-2f96-4483-9e92-513109dd37cb\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:54:54.479831 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.479044    6133 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/484e37a0-fd1d-4fd0-a200-939b8945984b-openshift-state-metrics-tls\") pod \"openshift-state-metrics-7bd7489957-f26w7\" (UID: \"484e37a0-fd1d-4fd0-a200-939b8945984b\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-f26w7"
Oct 30 21:54:54.479831 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.479116    6133 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/17da628d-31f5-456c-ad37-52c884d22cf4-volume-directive-shadow\") pod \"kube-state-metrics-5dc8d7d89b-8727r\" (UID: \"17da628d-31f5-456c-ad37-52c884d22cf4\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-8727r"
Oct 30 21:54:54.479831 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.479134    6133 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/446beacf-70a0-4d47-96ea-bbe1ca4a5f28-ovnkube-script-lib\") pod \"ovnkube-node-mn9vl\" (UID: \"446beacf-70a0-4d47-96ea-bbe1ca4a5f28\") " pod="openshift-ovn-kubernetes/ovnkube-node-mn9vl"
Oct 30 21:54:54.479831 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.479178    6133 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n87x8\" (UniqueName: \"kubernetes.io/projected/484e37a0-fd1d-4fd0-a200-939b8945984b-kube-api-access-n87x8\") pod \"openshift-state-metrics-7bd7489957-f26w7\" (UID: \"484e37a0-fd1d-4fd0-a200-939b8945984b\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-f26w7"
Oct 30 21:54:54.479831 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.479206    6133 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/17da628d-31f5-456c-ad37-52c884d22cf4-volume-directive-shadow\") pod \"kube-state-metrics-5dc8d7d89b-8727r\" (UID: \"17da628d-31f5-456c-ad37-52c884d22cf4\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-8727r"
Oct 30 21:54:54.479831 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.479229    6133 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/bd93518f-2f96-4483-9e92-513109dd37cb-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"bd93518f-2f96-4483-9e92-513109dd37cb\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:54:54.479831 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.479265    6133 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/17da628d-31f5-456c-ad37-52c884d22cf4-kube-state-metrics-tls\") pod \"kube-state-metrics-5dc8d7d89b-8727r\" (UID: \"17da628d-31f5-456c-ad37-52c884d22cf4\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-8727r"
Oct 30 21:54:54.479831 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.479324    6133 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/bd93518f-2f96-4483-9e92-513109dd37cb-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"bd93518f-2f96-4483-9e92-513109dd37cb\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:54:54.479831 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.479370    6133 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/17da628d-31f5-456c-ad37-52c884d22cf4-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-5dc8d7d89b-8727r\" (UID: \"17da628d-31f5-456c-ad37-52c884d22cf4\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-8727r"
Oct 30 21:54:54.479831 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.479417    6133 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/446beacf-70a0-4d47-96ea-bbe1ca4a5f28-host-kubelet\") pod \"ovnkube-node-mn9vl\" (UID: \"446beacf-70a0-4d47-96ea-bbe1ca4a5f28\") " pod="openshift-ovn-kubernetes/ovnkube-node-mn9vl"
Oct 30 21:54:54.479831 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.479483    6133 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/17da628d-31f5-456c-ad37-52c884d22cf4-kube-state-metrics-tls\") pod \"kube-state-metrics-5dc8d7d89b-8727r\" (UID: \"17da628d-31f5-456c-ad37-52c884d22cf4\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-8727r"
Oct 30 21:54:54.479831 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.479497    6133 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/209b4e58-e6c2-48f1-8589-f7b6be79720b-tmp-dir\") pod \"dns-default-fqmbc\" (UID: \"209b4e58-e6c2-48f1-8589-f7b6be79720b\") " pod="openshift-dns/dns-default-fqmbc"
Oct 30 21:54:54.479831 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.479529    6133 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/3e60fceb-8682-45cc-a472-f3d7017c3395-cni-binary-copy\") pod \"multus-additional-cni-plugins-c9tnl\" (UID: \"3e60fceb-8682-45cc-a472-f3d7017c3395\") " pod="openshift-multus/multus-additional-cni-plugins-c9tnl"
Oct 30 21:54:54.479831 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.479584    6133 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/209b4e58-e6c2-48f1-8589-f7b6be79720b-tmp-dir\") pod \"dns-default-fqmbc\" (UID: \"209b4e58-e6c2-48f1-8589-f7b6be79720b\") " pod="openshift-dns/dns-default-fqmbc"
Oct 30 21:54:54.479831 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.479578    6133 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/17da628d-31f5-456c-ad37-52c884d22cf4-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-5dc8d7d89b-8727r\" (UID: \"17da628d-31f5-456c-ad37-52c884d22cf4\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-8727r"
Oct 30 21:54:54.479831 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.479581    6133 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/bebdb584-1a96-453d-8a6d-b332e47bec3d-etc-systemd\") pod \"tuned-h8csb\" (UID: \"bebdb584-1a96-453d-8a6d-b332e47bec3d\") " pod="openshift-cluster-node-tuning-operator/tuned-h8csb"
Oct 30 21:54:54.479831 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.479617    6133 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/fbd8f09b-9962-4910-8b25-267cbbb3ea5d-secret-grpc-tls\") pod \"thanos-querier-868d6f78df-nx2k6\" (UID: \"fbd8f09b-9962-4910-8b25-267cbbb3ea5d\") " pod="openshift-monitoring/thanos-querier-868d6f78df-nx2k6"
Oct 30 21:54:54.479831 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.479674    6133 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/446beacf-70a0-4d47-96ea-bbe1ca4a5f28-ovn-node-metrics-cert\") pod \"ovnkube-node-mn9vl\" (UID: \"446beacf-70a0-4d47-96ea-bbe1ca4a5f28\") " pod="openshift-ovn-kubernetes/ovnkube-node-mn9vl"
Oct 30 21:54:54.479831 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.479721    6133 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/bebdb584-1a96-453d-8a6d-b332e47bec3d-sys\") pod \"tuned-h8csb\" (UID: \"bebdb584-1a96-453d-8a6d-b332e47bec3d\") " pod="openshift-cluster-node-tuning-operator/tuned-h8csb"
Oct 30 21:54:54.479831 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.479751    6133 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/3e60fceb-8682-45cc-a472-f3d7017c3395-cni-binary-copy\") pod \"multus-additional-cni-plugins-c9tnl\" (UID: \"3e60fceb-8682-45cc-a472-f3d7017c3395\") " pod="openshift-multus/multus-additional-cni-plugins-c9tnl"
Oct 30 21:54:54.479831 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.479821    6133 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/fbd8f09b-9962-4910-8b25-267cbbb3ea5d-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-868d6f78df-nx2k6\" (UID: \"fbd8f09b-9962-4910-8b25-267cbbb3ea5d\") " pod="openshift-monitoring/thanos-querier-868d6f78df-nx2k6"
Oct 30 21:54:54.479831 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.479825    6133 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/fbd8f09b-9962-4910-8b25-267cbbb3ea5d-secret-grpc-tls\") pod \"thanos-querier-868d6f78df-nx2k6\" (UID: \"fbd8f09b-9962-4910-8b25-267cbbb3ea5d\") " pod="openshift-monitoring/thanos-querier-868d6f78df-nx2k6"
Oct 30 21:54:54.479831 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.479851    6133 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fbd8f09b-9962-4910-8b25-267cbbb3ea5d-metrics-client-ca\") pod \"thanos-querier-868d6f78df-nx2k6\" (UID: \"fbd8f09b-9962-4910-8b25-267cbbb3ea5d\") " pod="openshift-monitoring/thanos-querier-868d6f78df-nx2k6"
Oct 30 21:54:54.481131 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.479876    6133 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fbd8f09b-9962-4910-8b25-267cbbb3ea5d-tmp\") pod \"thanos-querier-868d6f78df-nx2k6\" (UID: \"fbd8f09b-9962-4910-8b25-267cbbb3ea5d\") " pod="openshift-monitoring/thanos-querier-868d6f78df-nx2k6"
Oct 30 21:54:54.481131 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.479902    6133 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/446beacf-70a0-4d47-96ea-bbe1ca4a5f28-ovn-node-metrics-cert\") pod \"ovnkube-node-mn9vl\" (UID: \"446beacf-70a0-4d47-96ea-bbe1ca4a5f28\") " pod="openshift-ovn-kubernetes/ovnkube-node-mn9vl"
Oct 30 21:54:54.481131 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.479918    6133 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5d5057fc-dd8a-4dce-b8bb-70b2dc3b074c-host-run-netns\") pod \"multus-jkdxt\" (UID: \"5d5057fc-dd8a-4dce-b8bb-70b2dc3b074c\") " pod="openshift-multus/multus-jkdxt"
Oct 30 21:54:54.481131 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.479968    6133 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/bebdb584-1a96-453d-8a6d-b332e47bec3d-lib-modules\") pod \"tuned-h8csb\" (UID: \"bebdb584-1a96-453d-8a6d-b332e47bec3d\") " pod="openshift-cluster-node-tuning-operator/tuned-h8csb"
Oct 30 21:54:54.481131 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.480022    6133 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/fbd8f09b-9962-4910-8b25-267cbbb3ea5d-secret-thanos-querier-tls\") pod \"thanos-querier-868d6f78df-nx2k6\" (UID: \"fbd8f09b-9962-4910-8b25-267cbbb3ea5d\") " pod="openshift-monitoring/thanos-querier-868d6f78df-nx2k6"
Oct 30 21:54:54.481131 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.480054    6133 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/446beacf-70a0-4d47-96ea-bbe1ca4a5f28-host-cni-netd\") pod \"ovnkube-node-mn9vl\" (UID: \"446beacf-70a0-4d47-96ea-bbe1ca4a5f28\") " pod="openshift-ovn-kubernetes/ovnkube-node-mn9vl"
Oct 30 21:54:54.481131 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.480097    6133 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fbd8f09b-9962-4910-8b25-267cbbb3ea5d-metrics-client-ca\") pod \"thanos-querier-868d6f78df-nx2k6\" (UID: \"fbd8f09b-9962-4910-8b25-267cbbb3ea5d\") " pod="openshift-monitoring/thanos-querier-868d6f78df-nx2k6"
Oct 30 21:54:54.481131 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.480151    6133 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9f2kg\" (UniqueName: \"kubernetes.io/projected/4621048c-230e-441f-85fc-423e4ba525ec-kube-api-access-9f2kg\") pod \"ingress-canary-fl5mx\" (UID: \"4621048c-230e-441f-85fc-423e4ba525ec\") " pod="openshift-ingress-canary/ingress-canary-fl5mx"
Oct 30 21:54:54.481131 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.480181    6133 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bw6lg\" (UniqueName: \"kubernetes.io/projected/17da628d-31f5-456c-ad37-52c884d22cf4-kube-api-access-bw6lg\") pod \"kube-state-metrics-5dc8d7d89b-8727r\" (UID: \"17da628d-31f5-456c-ad37-52c884d22cf4\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-8727r"
Oct 30 21:54:54.481131 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.480207    6133 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/bebdb584-1a96-453d-8a6d-b332e47bec3d-etc-tuned\") pod \"tuned-h8csb\" (UID: \"bebdb584-1a96-453d-8a6d-b332e47bec3d\") " pod="openshift-cluster-node-tuning-operator/tuned-h8csb"
Oct 30 21:54:54.481131 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.480235    6133 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/446beacf-70a0-4d47-96ea-bbe1ca4a5f28-log-socket\") pod \"ovnkube-node-mn9vl\" (UID: \"446beacf-70a0-4d47-96ea-bbe1ca4a5f28\") " pod="openshift-ovn-kubernetes/ovnkube-node-mn9vl"
Oct 30 21:54:54.481131 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.480253    6133 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/fbd8f09b-9962-4910-8b25-267cbbb3ea5d-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-868d6f78df-nx2k6\" (UID: \"fbd8f09b-9962-4910-8b25-267cbbb3ea5d\") " pod="openshift-monitoring/thanos-querier-868d6f78df-nx2k6"
Oct 30 21:54:54.481131 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.480302    6133 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bd93518f-2f96-4483-9e92-513109dd37cb-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"bd93518f-2f96-4483-9e92-513109dd37cb\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:54:54.481131 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.480320    6133 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/bebdb584-1a96-453d-8a6d-b332e47bec3d-etc-tuned\") pod \"tuned-h8csb\" (UID: \"bebdb584-1a96-453d-8a6d-b332e47bec3d\") " pod="openshift-cluster-node-tuning-operator/tuned-h8csb"
Oct 30 21:54:54.481131 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.480330    6133 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/bd93518f-2f96-4483-9e92-513109dd37cb-web-config\") pod \"prometheus-k8s-0\" (UID: \"bd93518f-2f96-4483-9e92-513109dd37cb\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:54:54.481131 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.480355    6133 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/fc4a67d9-8c7c-4f71-9a8a-2bba09329bc6-hosts-file\") pod \"node-resolver-wwb9f\" (UID: \"fc4a67d9-8c7c-4f71-9a8a-2bba09329bc6\") " pod="openshift-dns/node-resolver-wwb9f"
Oct 30 21:54:54.481131 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.480384    6133 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/3e60fceb-8682-45cc-a472-f3d7017c3395-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-c9tnl\" (UID: \"3e60fceb-8682-45cc-a472-f3d7017c3395\") " pod="openshift-multus/multus-additional-cni-plugins-c9tnl"
Oct 30 21:54:54.481131 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.480411    6133 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nbqv6\" (UniqueName: \"kubernetes.io/projected/fe472529-1294-46a1-a666-dc00d5d37eca-kube-api-access-nbqv6\") pod \"node-ca-qzgkj\" (UID: \"fe472529-1294-46a1-a666-dc00d5d37eca\") " pod="openshift-image-registry/node-ca-qzgkj"
Oct 30 21:54:54.481131 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.480436    6133 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/bebdb584-1a96-453d-8a6d-b332e47bec3d-host\") pod \"tuned-h8csb\" (UID: \"bebdb584-1a96-453d-8a6d-b332e47bec3d\") " pod="openshift-cluster-node-tuning-operator/tuned-h8csb"
Oct 30 21:54:54.481131 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.480502    6133 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/fbd8f09b-9962-4910-8b25-267cbbb3ea5d-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-868d6f78df-nx2k6\" (UID: \"fbd8f09b-9962-4910-8b25-267cbbb3ea5d\") " pod="openshift-monitoring/thanos-querier-868d6f78df-nx2k6"
Oct 30 21:54:54.481131 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.480513    6133 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/bd93518f-2f96-4483-9e92-513109dd37cb-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"bd93518f-2f96-4483-9e92-513109dd37cb\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:54:54.481131 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.480529    6133 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/fbd8f09b-9962-4910-8b25-267cbbb3ea5d-secret-thanos-querier-tls\") pod \"thanos-querier-868d6f78df-nx2k6\" (UID: \"fbd8f09b-9962-4910-8b25-267cbbb3ea5d\") " pod="openshift-monitoring/thanos-querier-868d6f78df-nx2k6"
Oct 30 21:54:54.481131 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.480534    6133 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6192d2be-bba7-4099-b54c-e7e9b8d75389-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-6jppj\" (UID: \"6192d2be-bba7-4099-b54c-e7e9b8d75389\") " pod="openshift-insights/insights-runtime-extractor-6jppj"
Oct 30 21:54:54.481131 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.480585    6133 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bd93518f-2f96-4483-9e92-513109dd37cb-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"bd93518f-2f96-4483-9e92-513109dd37cb\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:54:54.481131 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.480613    6133 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bd93518f-2f96-4483-9e92-513109dd37cb-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"bd93518f-2f96-4483-9e92-513109dd37cb\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:54:54.481131 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.480638    6133 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fc4a67d9-8c7c-4f71-9a8a-2bba09329bc6-tmp-dir\") pod \"node-resolver-wwb9f\" (UID: \"fc4a67d9-8c7c-4f71-9a8a-2bba09329bc6\") " pod="openshift-dns/node-resolver-wwb9f"
Oct 30 21:54:54.481131 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.480661    6133 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7ddfa67d-20a0-473d-937f-25fd8e0ba80e-sys\") pod \"node-exporter-df5jt\" (UID: \"7ddfa67d-20a0-473d-937f-25fd8e0ba80e\") " pod="openshift-monitoring/node-exporter-df5jt"
Oct 30 21:54:54.481131 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.480688    6133 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/fbd8f09b-9962-4910-8b25-267cbbb3ea5d-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-868d6f78df-nx2k6\" (UID: \"fbd8f09b-9962-4910-8b25-267cbbb3ea5d\") " pod="openshift-monitoring/thanos-querier-868d6f78df-nx2k6"
Oct 30 21:54:54.481131 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.480690    6133 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bd93518f-2f96-4483-9e92-513109dd37cb-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"bd93518f-2f96-4483-9e92-513109dd37cb\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:54:54.481131 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.480901    6133 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/af83b188-09cb-4621-a9da-be33856d19f0-kubelet-config\") pod \"global-pull-secret-syncer-q7fwz\" (UID: \"af83b188-09cb-4621-a9da-be33856d19f0\") " pod="kube-system/global-pull-secret-syncer-q7fwz"
Oct 30 21:54:54.481131 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.480904    6133 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fc4a67d9-8c7c-4f71-9a8a-2bba09329bc6-tmp-dir\") pod \"node-resolver-wwb9f\" (UID: \"fc4a67d9-8c7c-4f71-9a8a-2bba09329bc6\") " pod="openshift-dns/node-resolver-wwb9f"
Oct 30 21:54:54.481131 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.480939    6133 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/446beacf-70a0-4d47-96ea-bbe1ca4a5f28-var-lib-openvswitch\") pod \"ovnkube-node-mn9vl\" (UID: \"446beacf-70a0-4d47-96ea-bbe1ca4a5f28\") " pod="openshift-ovn-kubernetes/ovnkube-node-mn9vl"
Oct 30 21:54:54.481131 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.481017    6133 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/3e60fceb-8682-45cc-a472-f3d7017c3395-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-c9tnl\" (UID: \"3e60fceb-8682-45cc-a472-f3d7017c3395\") " pod="openshift-multus/multus-additional-cni-plugins-c9tnl"
Oct 30 21:54:54.481131 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.481026    6133 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bd93518f-2f96-4483-9e92-513109dd37cb-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"bd93518f-2f96-4483-9e92-513109dd37cb\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:54:54.481131 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.481066    6133 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bd93518f-2f96-4483-9e92-513109dd37cb-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"bd93518f-2f96-4483-9e92-513109dd37cb\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:54:54.481131 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.481073    6133 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/bd93518f-2f96-4483-9e92-513109dd37cb-web-config\") pod \"prometheus-k8s-0\" (UID: \"bd93518f-2f96-4483-9e92-513109dd37cb\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:54:54.481131 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.481080    6133 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fbd8f09b-9962-4910-8b25-267cbbb3ea5d-tmp\") pod \"thanos-querier-868d6f78df-nx2k6\" (UID: \"fbd8f09b-9962-4910-8b25-267cbbb3ea5d\") " pod="openshift-monitoring/thanos-querier-868d6f78df-nx2k6"
Oct 30 21:54:54.481131 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.481073    6133 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6192d2be-bba7-4099-b54c-e7e9b8d75389-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-6jppj\" (UID: \"6192d2be-bba7-4099-b54c-e7e9b8d75389\") " pod="openshift-insights/insights-runtime-extractor-6jppj"
Oct 30 21:54:54.481131 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.481107    6133 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/fbd8f09b-9962-4910-8b25-267cbbb3ea5d-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-868d6f78df-nx2k6\" (UID: \"fbd8f09b-9962-4910-8b25-267cbbb3ea5d\") " pod="openshift-monitoring/thanos-querier-868d6f78df-nx2k6"
Oct 30 21:54:54.482267 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.481214    6133 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/fbd8f09b-9962-4910-8b25-267cbbb3ea5d-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-868d6f78df-nx2k6\" (UID: \"fbd8f09b-9962-4910-8b25-267cbbb3ea5d\") " pod="openshift-monitoring/thanos-querier-868d6f78df-nx2k6"
Oct 30 21:54:54.495727 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.495690    6133 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6bsvp\" (UniqueName: \"kubernetes.io/projected/3e60fceb-8682-45cc-a472-f3d7017c3395-kube-api-access-6bsvp\") pod \"multus-additional-cni-plugins-c9tnl\" (UID: \"3e60fceb-8682-45cc-a472-f3d7017c3395\") " pod="openshift-multus/multus-additional-cni-plugins-c9tnl"
Oct 30 21:54:54.495873 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.495693    6133 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mrtmr\" (UniqueName: \"kubernetes.io/projected/209b4e58-e6c2-48f1-8589-f7b6be79720b-kube-api-access-mrtmr\") pod \"dns-default-fqmbc\" (UID: \"209b4e58-e6c2-48f1-8589-f7b6be79720b\") " pod="openshift-dns/dns-default-fqmbc"
Oct 30 21:54:54.497150 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.497128    6133 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ljsqq\" (UniqueName: \"kubernetes.io/projected/32ae0d42-4b76-47b7-b9d6-26b09f00ba5d-kube-api-access-ljsqq\") pod \"network-check-target-fvbz8\" (UID: \"32ae0d42-4b76-47b7-b9d6-26b09f00ba5d\") " pod="openshift-network-diagnostics/network-check-target-fvbz8"
Oct 30 21:54:54.499452 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.499433    6133 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w78hr\" (UniqueName: \"kubernetes.io/projected/9d2cced5-9ceb-421f-b831-8370a5ee0ca9-kube-api-access-w78hr\") pod \"monitoring-plugin-77f4f76594-htsvq\" (UID: \"9d2cced5-9ceb-421f-b831-8370a5ee0ca9\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-htsvq"
Oct 30 21:54:54.499581 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.499564    6133 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wsnr5\" (UniqueName: \"kubernetes.io/projected/7ddfa67d-20a0-473d-937f-25fd8e0ba80e-kube-api-access-wsnr5\") pod \"node-exporter-df5jt\" (UID: \"7ddfa67d-20a0-473d-937f-25fd8e0ba80e\") " pod="openshift-monitoring/node-exporter-df5jt"
Oct 30 21:54:54.499694 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.499678    6133 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r4qz4\" (UniqueName: \"kubernetes.io/projected/bebdb584-1a96-453d-8a6d-b332e47bec3d-kube-api-access-r4qz4\") pod \"tuned-h8csb\" (UID: \"bebdb584-1a96-453d-8a6d-b332e47bec3d\") " pod="openshift-cluster-node-tuning-operator/tuned-h8csb"
Oct 30 21:54:54.499971 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.499954    6133 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kncjh\" (UniqueName: \"kubernetes.io/projected/5d5057fc-dd8a-4dce-b8bb-70b2dc3b074c-kube-api-access-kncjh\") pod \"multus-jkdxt\" (UID: \"5d5057fc-dd8a-4dce-b8bb-70b2dc3b074c\") " pod="openshift-multus/multus-jkdxt"
Oct 30 21:54:54.500315 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.500299    6133 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jw4wk\" (UniqueName: \"kubernetes.io/projected/6192d2be-bba7-4099-b54c-e7e9b8d75389-kube-api-access-jw4wk\") pod \"insights-runtime-extractor-6jppj\" (UID: \"6192d2be-bba7-4099-b54c-e7e9b8d75389\") " pod="openshift-insights/insights-runtime-extractor-6jppj"
Oct 30 21:54:54.500933 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.500916    6133 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h6b6r\" (UniqueName: \"kubernetes.io/projected/613f514c-6011-4936-8df5-a59b960cd3a6-kube-api-access-h6b6r\") pod \"iptables-alerter-kghfh\" (UID: \"613f514c-6011-4936-8df5-a59b960cd3a6\") " pod="openshift-network-operator/iptables-alerter-kghfh"
Oct 30 21:54:54.501425 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.501409    6133 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r6wtm\" (UniqueName: \"kubernetes.io/projected/532658b6-c86e-4feb-89cb-14a91316a7a0-kube-api-access-r6wtm\") pod \"network-metrics-daemon-n8xrh\" (UID: \"532658b6-c86e-4feb-89cb-14a91316a7a0\") " pod="openshift-multus/network-metrics-daemon-n8xrh"
Oct 30 21:54:54.501646 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.501631    6133 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qrt69\" (UniqueName: \"kubernetes.io/projected/bd93518f-2f96-4483-9e92-513109dd37cb-kube-api-access-qrt69\") pod \"prometheus-k8s-0\" (UID: \"bd93518f-2f96-4483-9e92-513109dd37cb\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:54:54.503793 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.503774    6133 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lgk4w\" (UniqueName: \"kubernetes.io/projected/15e1de31-92a1-4179-8ec1-400346880d1d-kube-api-access-lgk4w\") pod \"aws-ebs-csi-driver-node-kqbp9\" (UID: \"15e1de31-92a1-4179-8ec1-400346880d1d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kqbp9"
Oct 30 21:54:54.504176 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.504129    6133 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n87x8\" (UniqueName: \"kubernetes.io/projected/484e37a0-fd1d-4fd0-a200-939b8945984b-kube-api-access-n87x8\") pod \"openshift-state-metrics-7bd7489957-f26w7\" (UID: \"484e37a0-fd1d-4fd0-a200-939b8945984b\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-f26w7"
Oct 30 21:54:54.504246 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.504222    6133 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zzw6w\" (UniqueName: \"kubernetes.io/projected/fbd8f09b-9962-4910-8b25-267cbbb3ea5d-kube-api-access-zzw6w\") pod \"thanos-querier-868d6f78df-nx2k6\" (UID: \"fbd8f09b-9962-4910-8b25-267cbbb3ea5d\") " pod="openshift-monitoring/thanos-querier-868d6f78df-nx2k6"
Oct 30 21:54:54.504431 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.504415    6133 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9f2kg\" (UniqueName: \"kubernetes.io/projected/4621048c-230e-441f-85fc-423e4ba525ec-kube-api-access-9f2kg\") pod \"ingress-canary-fl5mx\" (UID: \"4621048c-230e-441f-85fc-423e4ba525ec\") " pod="openshift-ingress-canary/ingress-canary-fl5mx"
Oct 30 21:54:54.504602 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.504587    6133 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mkrsw\" (UniqueName: \"kubernetes.io/projected/446beacf-70a0-4d47-96ea-bbe1ca4a5f28-kube-api-access-mkrsw\") pod \"ovnkube-node-mn9vl\" (UID: \"446beacf-70a0-4d47-96ea-bbe1ca4a5f28\") " pod="openshift-ovn-kubernetes/ovnkube-node-mn9vl"
Oct 30 21:54:54.505477 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.505456    6133 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jp25c\" (UniqueName: \"kubernetes.io/projected/fc4a67d9-8c7c-4f71-9a8a-2bba09329bc6-kube-api-access-jp25c\") pod \"node-resolver-wwb9f\" (UID: \"fc4a67d9-8c7c-4f71-9a8a-2bba09329bc6\") " pod="openshift-dns/node-resolver-wwb9f"
Oct 30 21:54:54.505854 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.505838    6133 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bw6lg\" (UniqueName: \"kubernetes.io/projected/17da628d-31f5-456c-ad37-52c884d22cf4-kube-api-access-bw6lg\") pod \"kube-state-metrics-5dc8d7d89b-8727r\" (UID: \"17da628d-31f5-456c-ad37-52c884d22cf4\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-8727r"
Oct 30 21:54:54.505971 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.505956    6133 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nbqv6\" (UniqueName: \"kubernetes.io/projected/fe472529-1294-46a1-a666-dc00d5d37eca-kube-api-access-nbqv6\") pod \"node-ca-qzgkj\" (UID: \"fe472529-1294-46a1-a666-dc00d5d37eca\") " pod="openshift-image-registry/node-ca-qzgkj"
Oct 30 21:54:54.582046 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.581940    6133 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/15e1de31-92a1-4179-8ec1-400346880d1d-etc-selinux\") pod \"aws-ebs-csi-driver-node-kqbp9\" (UID: \"15e1de31-92a1-4179-8ec1-400346880d1d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kqbp9"
Oct 30 21:54:54.582046 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.581981    6133 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/446beacf-70a0-4d47-96ea-bbe1ca4a5f28-run-ovn\") pod \"ovnkube-node-mn9vl\" (UID: \"446beacf-70a0-4d47-96ea-bbe1ca4a5f28\") " pod="openshift-ovn-kubernetes/ovnkube-node-mn9vl"
Oct 30 21:54:54.582046 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.582014    6133 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/6192d2be-bba7-4099-b54c-e7e9b8d75389-crio-socket\") pod \"insights-runtime-extractor-6jppj\" (UID: \"6192d2be-bba7-4099-b54c-e7e9b8d75389\") " pod="openshift-insights/insights-runtime-extractor-6jppj"
Oct 30 21:54:54.582046 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.582037    6133 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/3e60fceb-8682-45cc-a472-f3d7017c3395-tuning-conf-dir\") pod \"multus-additional-cni-plugins-c9tnl\" (UID: \"3e60fceb-8682-45cc-a472-f3d7017c3395\") " pod="openshift-multus/multus-additional-cni-plugins-c9tnl"
Oct 30 21:54:54.582381 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.582057    6133 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/15e1de31-92a1-4179-8ec1-400346880d1d-kubelet-dir\") pod \"aws-ebs-csi-driver-node-kqbp9\" (UID: \"15e1de31-92a1-4179-8ec1-400346880d1d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kqbp9"
Oct 30 21:54:54.582381 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.582071    6133 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/bebdb584-1a96-453d-8a6d-b332e47bec3d-etc-sysctl-d\") pod \"tuned-h8csb\" (UID: \"bebdb584-1a96-453d-8a6d-b332e47bec3d\") " pod="openshift-cluster-node-tuning-operator/tuned-h8csb"
Oct 30 21:54:54.582381 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.582088    6133 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5d5057fc-dd8a-4dce-b8bb-70b2dc3b074c-host-var-lib-cni-bin\") pod \"multus-jkdxt\" (UID: \"5d5057fc-dd8a-4dce-b8bb-70b2dc3b074c\") " pod="openshift-multus/multus-jkdxt"
Oct 30 21:54:54.582381 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.582119    6133 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/3e60fceb-8682-45cc-a472-f3d7017c3395-tuning-conf-dir\") pod \"multus-additional-cni-plugins-c9tnl\" (UID: \"3e60fceb-8682-45cc-a472-f3d7017c3395\") " pod="openshift-multus/multus-additional-cni-plugins-c9tnl"
Oct 30 21:54:54.582381 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.582127    6133 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/6192d2be-bba7-4099-b54c-e7e9b8d75389-crio-socket\") pod \"insights-runtime-extractor-6jppj\" (UID: \"6192d2be-bba7-4099-b54c-e7e9b8d75389\") " pod="openshift-insights/insights-runtime-extractor-6jppj"
Oct 30 21:54:54.582381 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.582136    6133 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/446beacf-70a0-4d47-96ea-bbe1ca4a5f28-run-ovn\") pod \"ovnkube-node-mn9vl\" (UID: \"446beacf-70a0-4d47-96ea-bbe1ca4a5f28\") " pod="openshift-ovn-kubernetes/ovnkube-node-mn9vl"
Oct 30 21:54:54.582381 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.582128    6133 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5d5057fc-dd8a-4dce-b8bb-70b2dc3b074c-host-var-lib-cni-bin\") pod \"multus-jkdxt\" (UID: \"5d5057fc-dd8a-4dce-b8bb-70b2dc3b074c\") " pod="openshift-multus/multus-jkdxt"
Oct 30 21:54:54.582381 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.582068    6133 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/15e1de31-92a1-4179-8ec1-400346880d1d-etc-selinux\") pod \"aws-ebs-csi-driver-node-kqbp9\" (UID: \"15e1de31-92a1-4179-8ec1-400346880d1d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kqbp9"
Oct 30 21:54:54.582381 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.582171    6133 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5d5057fc-dd8a-4dce-b8bb-70b2dc3b074c-system-cni-dir\") pod \"multus-jkdxt\" (UID: \"5d5057fc-dd8a-4dce-b8bb-70b2dc3b074c\") " pod="openshift-multus/multus-jkdxt"
Oct 30 21:54:54.582381 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.582183    6133 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/15e1de31-92a1-4179-8ec1-400346880d1d-kubelet-dir\") pod \"aws-ebs-csi-driver-node-kqbp9\" (UID: \"15e1de31-92a1-4179-8ec1-400346880d1d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kqbp9"
Oct 30 21:54:54.582381 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.582200    6133 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/5d5057fc-dd8a-4dce-b8bb-70b2dc3b074c-host-run-k8s-cni-cncf-io\") pod \"multus-jkdxt\" (UID: \"5d5057fc-dd8a-4dce-b8bb-70b2dc3b074c\") " pod="openshift-multus/multus-jkdxt"
Oct 30 21:54:54.582381 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.582224    6133 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bebdb584-1a96-453d-8a6d-b332e47bec3d-etc-kubernetes\") pod \"tuned-h8csb\" (UID: \"bebdb584-1a96-453d-8a6d-b332e47bec3d\") " pod="openshift-cluster-node-tuning-operator/tuned-h8csb"
Oct 30 21:54:54.582381 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.582226    6133 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/bebdb584-1a96-453d-8a6d-b332e47bec3d-etc-sysctl-d\") pod \"tuned-h8csb\" (UID: \"bebdb584-1a96-453d-8a6d-b332e47bec3d\") " pod="openshift-cluster-node-tuning-operator/tuned-h8csb"
Oct 30 21:54:54.582381 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.582228    6133 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5d5057fc-dd8a-4dce-b8bb-70b2dc3b074c-system-cni-dir\") pod \"multus-jkdxt\" (UID: \"5d5057fc-dd8a-4dce-b8bb-70b2dc3b074c\") " pod="openshift-multus/multus-jkdxt"
Oct 30 21:54:54.582381 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.582249    6133 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/15e1de31-92a1-4179-8ec1-400346880d1d-device-dir\") pod \"aws-ebs-csi-driver-node-kqbp9\" (UID: \"15e1de31-92a1-4179-8ec1-400346880d1d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kqbp9"
Oct 30 21:54:54.582381 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.582261    6133 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/5d5057fc-dd8a-4dce-b8bb-70b2dc3b074c-host-run-k8s-cni-cncf-io\") pod \"multus-jkdxt\" (UID: \"5d5057fc-dd8a-4dce-b8bb-70b2dc3b074c\") " pod="openshift-multus/multus-jkdxt"
Oct 30 21:54:54.582381 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.582275    6133 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3e60fceb-8682-45cc-a472-f3d7017c3395-system-cni-dir\") pod \"multus-additional-cni-plugins-c9tnl\" (UID: \"3e60fceb-8682-45cc-a472-f3d7017c3395\") " pod="openshift-multus/multus-additional-cni-plugins-c9tnl"
Oct 30 21:54:54.582381 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.582282    6133 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bebdb584-1a96-453d-8a6d-b332e47bec3d-etc-kubernetes\") pod \"tuned-h8csb\" (UID: \"bebdb584-1a96-453d-8a6d-b332e47bec3d\") " pod="openshift-cluster-node-tuning-operator/tuned-h8csb"
Oct 30 21:54:54.582381 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.582284    6133 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/15e1de31-92a1-4179-8ec1-400346880d1d-device-dir\") pod \"aws-ebs-csi-driver-node-kqbp9\" (UID: \"15e1de31-92a1-4179-8ec1-400346880d1d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kqbp9"
Oct 30 21:54:54.582381 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.582303    6133 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5d5057fc-dd8a-4dce-b8bb-70b2dc3b074c-cnibin\") pod \"multus-jkdxt\" (UID: \"5d5057fc-dd8a-4dce-b8bb-70b2dc3b074c\") " pod="openshift-multus/multus-jkdxt"
Oct 30 21:54:54.582381 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.582329    6133 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/446beacf-70a0-4d47-96ea-bbe1ca4a5f28-systemd-units\") pod \"ovnkube-node-mn9vl\" (UID: \"446beacf-70a0-4d47-96ea-bbe1ca4a5f28\") " pod="openshift-ovn-kubernetes/ovnkube-node-mn9vl"
Oct 30 21:54:54.582381 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.582335    6133 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3e60fceb-8682-45cc-a472-f3d7017c3395-system-cni-dir\") pod \"multus-additional-cni-plugins-c9tnl\" (UID: \"3e60fceb-8682-45cc-a472-f3d7017c3395\") " pod="openshift-multus/multus-additional-cni-plugins-c9tnl"
Oct 30 21:54:54.582381 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.582357    6133 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5d5057fc-dd8a-4dce-b8bb-70b2dc3b074c-multus-conf-dir\") pod \"multus-jkdxt\" (UID: \"5d5057fc-dd8a-4dce-b8bb-70b2dc3b074c\") " pod="openshift-multus/multus-jkdxt"
Oct 30 21:54:54.582381 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.582369    6133 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5d5057fc-dd8a-4dce-b8bb-70b2dc3b074c-cnibin\") pod \"multus-jkdxt\" (UID: \"5d5057fc-dd8a-4dce-b8bb-70b2dc3b074c\") " pod="openshift-multus/multus-jkdxt"
Oct 30 21:54:54.582381 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.582382    6133 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/bebdb584-1a96-453d-8a6d-b332e47bec3d-etc-sysctl-conf\") pod \"tuned-h8csb\" (UID: \"bebdb584-1a96-453d-8a6d-b332e47bec3d\") " pod="openshift-cluster-node-tuning-operator/tuned-h8csb"
Oct 30 21:54:54.583437 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.582406    6133 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/446beacf-70a0-4d47-96ea-bbe1ca4a5f28-host-run-ovn-kubernetes\") pod \"ovnkube-node-mn9vl\" (UID: \"446beacf-70a0-4d47-96ea-bbe1ca4a5f28\") " pod="openshift-ovn-kubernetes/ovnkube-node-mn9vl"
Oct 30 21:54:54.583437 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.582425    6133 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5d5057fc-dd8a-4dce-b8bb-70b2dc3b074c-multus-conf-dir\") pod \"multus-jkdxt\" (UID: \"5d5057fc-dd8a-4dce-b8bb-70b2dc3b074c\") " pod="openshift-multus/multus-jkdxt"
Oct 30 21:54:54.583437 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.582436    6133 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/bebdb584-1a96-453d-8a6d-b332e47bec3d-var-lib-kubelet\") pod \"tuned-h8csb\" (UID: \"bebdb584-1a96-453d-8a6d-b332e47bec3d\") " pod="openshift-cluster-node-tuning-operator/tuned-h8csb"
Oct 30 21:54:54.583437 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.582446    6133 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/446beacf-70a0-4d47-96ea-bbe1ca4a5f28-systemd-units\") pod \"ovnkube-node-mn9vl\" (UID: \"446beacf-70a0-4d47-96ea-bbe1ca4a5f28\") " pod="openshift-ovn-kubernetes/ovnkube-node-mn9vl"
Oct 30 21:54:54.583437 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.582460    6133 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5d5057fc-dd8a-4dce-b8bb-70b2dc3b074c-multus-cni-dir\") pod \"multus-jkdxt\" (UID: \"5d5057fc-dd8a-4dce-b8bb-70b2dc3b074c\") " pod="openshift-multus/multus-jkdxt"
Oct 30 21:54:54.583437 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.582465    6133 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/446beacf-70a0-4d47-96ea-bbe1ca4a5f28-host-run-ovn-kubernetes\") pod \"ovnkube-node-mn9vl\" (UID: \"446beacf-70a0-4d47-96ea-bbe1ca4a5f28\") " pod="openshift-ovn-kubernetes/ovnkube-node-mn9vl"
Oct 30 21:54:54.583437 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.582470    6133 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/bebdb584-1a96-453d-8a6d-b332e47bec3d-var-lib-kubelet\") pod \"tuned-h8csb\" (UID: \"bebdb584-1a96-453d-8a6d-b332e47bec3d\") " pod="openshift-cluster-node-tuning-operator/tuned-h8csb"
Oct 30 21:54:54.583437 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.582485    6133 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/15e1de31-92a1-4179-8ec1-400346880d1d-registration-dir\") pod \"aws-ebs-csi-driver-node-kqbp9\" (UID: \"15e1de31-92a1-4179-8ec1-400346880d1d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kqbp9"
Oct 30 21:54:54.583437 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.582505    6133 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/bebdb584-1a96-453d-8a6d-b332e47bec3d-etc-sysctl-conf\") pod \"tuned-h8csb\" (UID: \"bebdb584-1a96-453d-8a6d-b332e47bec3d\") " pod="openshift-cluster-node-tuning-operator/tuned-h8csb"
Oct 30 21:54:54.583437 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.582509    6133 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/15e1de31-92a1-4179-8ec1-400346880d1d-sys-fs\") pod \"aws-ebs-csi-driver-node-kqbp9\" (UID: \"15e1de31-92a1-4179-8ec1-400346880d1d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kqbp9"
Oct 30 21:54:54.583437 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.582542    6133 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/bebdb584-1a96-453d-8a6d-b332e47bec3d-run\") pod \"tuned-h8csb\" (UID: \"bebdb584-1a96-453d-8a6d-b332e47bec3d\") " pod="openshift-cluster-node-tuning-operator/tuned-h8csb"
Oct 30 21:54:54.583437 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.582554    6133 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/15e1de31-92a1-4179-8ec1-400346880d1d-registration-dir\") pod \"aws-ebs-csi-driver-node-kqbp9\" (UID: \"15e1de31-92a1-4179-8ec1-400346880d1d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kqbp9"
Oct 30 21:54:54.583437 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.582560    6133 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/446beacf-70a0-4d47-96ea-bbe1ca4a5f28-host-slash\") pod \"ovnkube-node-mn9vl\" (UID: \"446beacf-70a0-4d47-96ea-bbe1ca4a5f28\") " pod="openshift-ovn-kubernetes/ovnkube-node-mn9vl"
Oct 30 21:54:54.583437 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.582566    6133 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/bebdb584-1a96-453d-8a6d-b332e47bec3d-run\") pod \"tuned-h8csb\" (UID: \"bebdb584-1a96-453d-8a6d-b332e47bec3d\") " pod="openshift-cluster-node-tuning-operator/tuned-h8csb"
Oct 30 21:54:54.583437 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.582569    6133 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/15e1de31-92a1-4179-8ec1-400346880d1d-sys-fs\") pod \"aws-ebs-csi-driver-node-kqbp9\" (UID: \"15e1de31-92a1-4179-8ec1-400346880d1d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kqbp9"
Oct 30 21:54:54.583437 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.582580    6133 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/446beacf-70a0-4d47-96ea-bbe1ca4a5f28-node-log\") pod \"ovnkube-node-mn9vl\" (UID: \"446beacf-70a0-4d47-96ea-bbe1ca4a5f28\") " pod="openshift-ovn-kubernetes/ovnkube-node-mn9vl"
Oct 30 21:54:54.583437 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.582539    6133 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5d5057fc-dd8a-4dce-b8bb-70b2dc3b074c-multus-cni-dir\") pod \"multus-jkdxt\" (UID: \"5d5057fc-dd8a-4dce-b8bb-70b2dc3b074c\") " pod="openshift-multus/multus-jkdxt"
Oct 30 21:54:54.583437 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.582596    6133 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/5d5057fc-dd8a-4dce-b8bb-70b2dc3b074c-host-run-multus-certs\") pod \"multus-jkdxt\" (UID: \"5d5057fc-dd8a-4dce-b8bb-70b2dc3b074c\") " pod="openshift-multus/multus-jkdxt"
Oct 30 21:54:54.583437 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.582611    6133 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/bebdb584-1a96-453d-8a6d-b332e47bec3d-etc-modprobe-d\") pod \"tuned-h8csb\" (UID: \"bebdb584-1a96-453d-8a6d-b332e47bec3d\") " pod="openshift-cluster-node-tuning-operator/tuned-h8csb"
Oct 30 21:54:54.583437 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.582622    6133 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/5d5057fc-dd8a-4dce-b8bb-70b2dc3b074c-host-run-multus-certs\") pod \"multus-jkdxt\" (UID: \"5d5057fc-dd8a-4dce-b8bb-70b2dc3b074c\") " pod="openshift-multus/multus-jkdxt"
Oct 30 21:54:54.583437 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.582656    6133 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/fe472529-1294-46a1-a666-dc00d5d37eca-host\") pod \"node-ca-qzgkj\" (UID: \"fe472529-1294-46a1-a666-dc00d5d37eca\") " pod="openshift-image-registry/node-ca-qzgkj"
Oct 30 21:54:54.583437 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.582624    6133 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/446beacf-70a0-4d47-96ea-bbe1ca4a5f28-node-log\") pod \"ovnkube-node-mn9vl\" (UID: \"446beacf-70a0-4d47-96ea-bbe1ca4a5f28\") " pod="openshift-ovn-kubernetes/ovnkube-node-mn9vl"
Oct 30 21:54:54.583437 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.582595    6133 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/446beacf-70a0-4d47-96ea-bbe1ca4a5f28-host-slash\") pod \"ovnkube-node-mn9vl\" (UID: \"446beacf-70a0-4d47-96ea-bbe1ca4a5f28\") " pod="openshift-ovn-kubernetes/ovnkube-node-mn9vl"
Oct 30 21:54:54.583437 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.582626    6133 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/fe472529-1294-46a1-a666-dc00d5d37eca-host\") pod \"node-ca-qzgkj\" (UID: \"fe472529-1294-46a1-a666-dc00d5d37eca\") " pod="openshift-image-registry/node-ca-qzgkj"
Oct 30 21:54:54.583437 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.582692    6133 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/446beacf-70a0-4d47-96ea-bbe1ca4a5f28-host-cni-bin\") pod \"ovnkube-node-mn9vl\" (UID: \"446beacf-70a0-4d47-96ea-bbe1ca4a5f28\") " pod="openshift-ovn-kubernetes/ovnkube-node-mn9vl"
Oct 30 21:54:54.583437 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.582715    6133 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/5d5057fc-dd8a-4dce-b8bb-70b2dc3b074c-multus-socket-dir-parent\") pod \"multus-jkdxt\" (UID: \"5d5057fc-dd8a-4dce-b8bb-70b2dc3b074c\") " pod="openshift-multus/multus-jkdxt"
Oct 30 21:54:54.583437 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.582731    6133 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/446beacf-70a0-4d47-96ea-bbe1ca4a5f28-etc-openvswitch\") pod \"ovnkube-node-mn9vl\" (UID: \"446beacf-70a0-4d47-96ea-bbe1ca4a5f28\") " pod="openshift-ovn-kubernetes/ovnkube-node-mn9vl"
Oct 30 21:54:54.583437 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.582749    6133 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/15e1de31-92a1-4179-8ec1-400346880d1d-socket-dir\") pod \"aws-ebs-csi-driver-node-kqbp9\" (UID: \"15e1de31-92a1-4179-8ec1-400346880d1d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kqbp9"
Oct 30 21:54:54.583437 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.582756    6133 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/5d5057fc-dd8a-4dce-b8bb-70b2dc3b074c-multus-socket-dir-parent\") pod \"multus-jkdxt\" (UID: \"5d5057fc-dd8a-4dce-b8bb-70b2dc3b074c\") " pod="openshift-multus/multus-jkdxt"
Oct 30 21:54:54.583437 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.582768    6133 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/446beacf-70a0-4d47-96ea-bbe1ca4a5f28-etc-openvswitch\") pod \"ovnkube-node-mn9vl\" (UID: \"446beacf-70a0-4d47-96ea-bbe1ca4a5f28\") " pod="openshift-ovn-kubernetes/ovnkube-node-mn9vl"
Oct 30 21:54:54.583437 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.582772    6133 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/446beacf-70a0-4d47-96ea-bbe1ca4a5f28-host-kubelet\") pod \"ovnkube-node-mn9vl\" (UID: \"446beacf-70a0-4d47-96ea-bbe1ca4a5f28\") " pod="openshift-ovn-kubernetes/ovnkube-node-mn9vl"
Oct 30 21:54:54.583437 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.582714    6133 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/bebdb584-1a96-453d-8a6d-b332e47bec3d-etc-modprobe-d\") pod \"tuned-h8csb\" (UID: \"bebdb584-1a96-453d-8a6d-b332e47bec3d\") " pod="openshift-cluster-node-tuning-operator/tuned-h8csb"
Oct 30 21:54:54.583437 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.582715    6133 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/446beacf-70a0-4d47-96ea-bbe1ca4a5f28-host-cni-bin\") pod \"ovnkube-node-mn9vl\" (UID: \"446beacf-70a0-4d47-96ea-bbe1ca4a5f28\") " pod="openshift-ovn-kubernetes/ovnkube-node-mn9vl"
Oct 30 21:54:54.583437 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.582802    6133 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/446beacf-70a0-4d47-96ea-bbe1ca4a5f28-host-kubelet\") pod \"ovnkube-node-mn9vl\" (UID: \"446beacf-70a0-4d47-96ea-bbe1ca4a5f28\") " pod="openshift-ovn-kubernetes/ovnkube-node-mn9vl"
Oct 30 21:54:54.583437 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.582809    6133 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/15e1de31-92a1-4179-8ec1-400346880d1d-socket-dir\") pod \"aws-ebs-csi-driver-node-kqbp9\" (UID: \"15e1de31-92a1-4179-8ec1-400346880d1d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kqbp9"
Oct 30 21:54:54.583437 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.582821    6133 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/bebdb584-1a96-453d-8a6d-b332e47bec3d-etc-systemd\") pod \"tuned-h8csb\" (UID: \"bebdb584-1a96-453d-8a6d-b332e47bec3d\") " pod="openshift-cluster-node-tuning-operator/tuned-h8csb"
Oct 30 21:54:54.583437 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.582839    6133 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/bebdb584-1a96-453d-8a6d-b332e47bec3d-sys\") pod \"tuned-h8csb\" (UID: \"bebdb584-1a96-453d-8a6d-b332e47bec3d\") " pod="openshift-cluster-node-tuning-operator/tuned-h8csb"
Oct 30 21:54:54.583437 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.582847    6133 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/bebdb584-1a96-453d-8a6d-b332e47bec3d-etc-systemd\") pod \"tuned-h8csb\" (UID: \"bebdb584-1a96-453d-8a6d-b332e47bec3d\") " pod="openshift-cluster-node-tuning-operator/tuned-h8csb"
Oct 30 21:54:54.583437 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.582856    6133 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5d5057fc-dd8a-4dce-b8bb-70b2dc3b074c-host-run-netns\") pod \"multus-jkdxt\" (UID: \"5d5057fc-dd8a-4dce-b8bb-70b2dc3b074c\") " pod="openshift-multus/multus-jkdxt"
Oct 30 21:54:54.583437 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.582873    6133 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5d5057fc-dd8a-4dce-b8bb-70b2dc3b074c-host-run-netns\") pod \"multus-jkdxt\" (UID: \"5d5057fc-dd8a-4dce-b8bb-70b2dc3b074c\") " pod="openshift-multus/multus-jkdxt"
Oct 30 21:54:54.583437 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.582875    6133 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/bebdb584-1a96-453d-8a6d-b332e47bec3d-lib-modules\") pod \"tuned-h8csb\" (UID: \"bebdb584-1a96-453d-8a6d-b332e47bec3d\") " pod="openshift-cluster-node-tuning-operator/tuned-h8csb"
Oct 30 21:54:54.583437 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.582878    6133 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/bebdb584-1a96-453d-8a6d-b332e47bec3d-sys\") pod \"tuned-h8csb\" (UID: \"bebdb584-1a96-453d-8a6d-b332e47bec3d\") " pod="openshift-cluster-node-tuning-operator/tuned-h8csb"
Oct 30 21:54:54.583437 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.582899    6133 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/446beacf-70a0-4d47-96ea-bbe1ca4a5f28-host-cni-netd\") pod \"ovnkube-node-mn9vl\" (UID: \"446beacf-70a0-4d47-96ea-bbe1ca4a5f28\") " pod="openshift-ovn-kubernetes/ovnkube-node-mn9vl"
Oct 30 21:54:54.583437 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.582916    6133 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/446beacf-70a0-4d47-96ea-bbe1ca4a5f28-log-socket\") pod \"ovnkube-node-mn9vl\" (UID: \"446beacf-70a0-4d47-96ea-bbe1ca4a5f28\") " pod="openshift-ovn-kubernetes/ovnkube-node-mn9vl"
Oct 30 21:54:54.583437 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.582931    6133 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/fc4a67d9-8c7c-4f71-9a8a-2bba09329bc6-hosts-file\") pod \"node-resolver-wwb9f\" (UID: \"fc4a67d9-8c7c-4f71-9a8a-2bba09329bc6\") " pod="openshift-dns/node-resolver-wwb9f"
Oct 30 21:54:54.583437 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.582940    6133 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/bebdb584-1a96-453d-8a6d-b332e47bec3d-lib-modules\") pod \"tuned-h8csb\" (UID: \"bebdb584-1a96-453d-8a6d-b332e47bec3d\") " pod="openshift-cluster-node-tuning-operator/tuned-h8csb"
Oct 30 21:54:54.583437 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.582955    6133 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/446beacf-70a0-4d47-96ea-bbe1ca4a5f28-host-cni-netd\") pod \"ovnkube-node-mn9vl\" (UID: \"446beacf-70a0-4d47-96ea-bbe1ca4a5f28\") " pod="openshift-ovn-kubernetes/ovnkube-node-mn9vl"
Oct 30 21:54:54.583437 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.582962    6133 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/446beacf-70a0-4d47-96ea-bbe1ca4a5f28-log-socket\") pod \"ovnkube-node-mn9vl\" (UID: \"446beacf-70a0-4d47-96ea-bbe1ca4a5f28\") " pod="openshift-ovn-kubernetes/ovnkube-node-mn9vl"
Oct 30 21:54:54.583437 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.582969    6133 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/fc4a67d9-8c7c-4f71-9a8a-2bba09329bc6-hosts-file\") pod \"node-resolver-wwb9f\" (UID: \"fc4a67d9-8c7c-4f71-9a8a-2bba09329bc6\") " pod="openshift-dns/node-resolver-wwb9f"
Oct 30 21:54:54.583437 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.582972    6133 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/bebdb584-1a96-453d-8a6d-b332e47bec3d-host\") pod \"tuned-h8csb\" (UID: \"bebdb584-1a96-453d-8a6d-b332e47bec3d\") " pod="openshift-cluster-node-tuning-operator/tuned-h8csb"
Oct 30 21:54:54.583437 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.583002    6133 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7ddfa67d-20a0-473d-937f-25fd8e0ba80e-sys\") pod \"node-exporter-df5jt\" (UID: \"7ddfa67d-20a0-473d-937f-25fd8e0ba80e\") " pod="openshift-monitoring/node-exporter-df5jt"
Oct 30 21:54:54.583437 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.583024    6133 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/af83b188-09cb-4621-a9da-be33856d19f0-kubelet-config\") pod \"global-pull-secret-syncer-q7fwz\" (UID: \"af83b188-09cb-4621-a9da-be33856d19f0\") " pod="kube-system/global-pull-secret-syncer-q7fwz"
Oct 30 21:54:54.583437 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.583008    6133 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/bebdb584-1a96-453d-8a6d-b332e47bec3d-host\") pod \"tuned-h8csb\" (UID: \"bebdb584-1a96-453d-8a6d-b332e47bec3d\") " pod="openshift-cluster-node-tuning-operator/tuned-h8csb"
Oct 30 21:54:54.583437 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.583041    6133 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/446beacf-70a0-4d47-96ea-bbe1ca4a5f28-var-lib-openvswitch\") pod \"ovnkube-node-mn9vl\" (UID: \"446beacf-70a0-4d47-96ea-bbe1ca4a5f28\") " pod="openshift-ovn-kubernetes/ovnkube-node-mn9vl"
Oct 30 21:54:54.583437 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.583054    6133 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7ddfa67d-20a0-473d-937f-25fd8e0ba80e-sys\") pod \"node-exporter-df5jt\" (UID: \"7ddfa67d-20a0-473d-937f-25fd8e0ba80e\") " pod="openshift-monitoring/node-exporter-df5jt"
Oct 30 21:54:54.583437 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.583074    6133 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/af83b188-09cb-4621-a9da-be33856d19f0-kubelet-config\") pod \"global-pull-secret-syncer-q7fwz\" (UID: \"af83b188-09cb-4621-a9da-be33856d19f0\") " pod="kube-system/global-pull-secret-syncer-q7fwz"
Oct 30 21:54:54.583437 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.583066    6133 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/5d5057fc-dd8a-4dce-b8bb-70b2dc3b074c-host-var-lib-cni-multus\") pod \"multus-jkdxt\" (UID: \"5d5057fc-dd8a-4dce-b8bb-70b2dc3b074c\") " pod="openshift-multus/multus-jkdxt"
Oct 30 21:54:54.583437 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.583088    6133 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/5d5057fc-dd8a-4dce-b8bb-70b2dc3b074c-host-var-lib-cni-multus\") pod \"multus-jkdxt\" (UID: \"5d5057fc-dd8a-4dce-b8bb-70b2dc3b074c\") " pod="openshift-multus/multus-jkdxt"
Oct 30 21:54:54.583437 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.583099    6133 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/af83b188-09cb-4621-a9da-be33856d19f0-dbus\") pod \"global-pull-secret-syncer-q7fwz\" (UID: \"af83b188-09cb-4621-a9da-be33856d19f0\") " pod="kube-system/global-pull-secret-syncer-q7fwz"
Oct 30 21:54:54.583437 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.583099    6133 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/446beacf-70a0-4d47-96ea-bbe1ca4a5f28-var-lib-openvswitch\") pod \"ovnkube-node-mn9vl\" (UID: \"446beacf-70a0-4d47-96ea-bbe1ca4a5f28\") " pod="openshift-ovn-kubernetes/ovnkube-node-mn9vl"
Oct 30 21:54:54.583437 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.583116    6133 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/7ddfa67d-20a0-473d-937f-25fd8e0ba80e-node-exporter-wtmp\") pod \"node-exporter-df5jt\" (UID: \"7ddfa67d-20a0-473d-937f-25fd8e0ba80e\") " pod="openshift-monitoring/node-exporter-df5jt"
Oct 30 21:54:54.583437 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.583149    6133 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/af83b188-09cb-4621-a9da-be33856d19f0-dbus\") pod \"global-pull-secret-syncer-q7fwz\" (UID: \"af83b188-09cb-4621-a9da-be33856d19f0\") " pod="kube-system/global-pull-secret-syncer-q7fwz"
Oct 30 21:54:54.583437 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.583153    6133 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/446beacf-70a0-4d47-96ea-bbe1ca4a5f28-run-systemd\") pod \"ovnkube-node-mn9vl\" (UID: \"446beacf-70a0-4d47-96ea-bbe1ca4a5f28\") " pod="openshift-ovn-kubernetes/ovnkube-node-mn9vl"
Oct 30 21:54:54.583437 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.583180    6133 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/7ddfa67d-20a0-473d-937f-25fd8e0ba80e-node-exporter-wtmp\") pod \"node-exporter-df5jt\" (UID: \"7ddfa67d-20a0-473d-937f-25fd8e0ba80e\") " pod="openshift-monitoring/node-exporter-df5jt"
Oct 30 21:54:54.583437 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.583182    6133 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/5d5057fc-dd8a-4dce-b8bb-70b2dc3b074c-hostroot\") pod \"multus-jkdxt\" (UID: \"5d5057fc-dd8a-4dce-b8bb-70b2dc3b074c\") " pod="openshift-multus/multus-jkdxt"
Oct 30 21:54:54.583437 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.583192    6133 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/446beacf-70a0-4d47-96ea-bbe1ca4a5f28-run-systemd\") pod \"ovnkube-node-mn9vl\" (UID: \"446beacf-70a0-4d47-96ea-bbe1ca4a5f28\") " pod="openshift-ovn-kubernetes/ovnkube-node-mn9vl"
Oct 30 21:54:54.583437 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.583207    6133 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/bebdb584-1a96-453d-8a6d-b332e47bec3d-etc-sysconfig\") pod \"tuned-h8csb\" (UID: \"bebdb584-1a96-453d-8a6d-b332e47bec3d\") " pod="openshift-cluster-node-tuning-operator/tuned-h8csb"
Oct 30 21:54:54.583437 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.583235    6133 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/446beacf-70a0-4d47-96ea-bbe1ca4a5f28-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-mn9vl\" (UID: \"446beacf-70a0-4d47-96ea-bbe1ca4a5f28\") " pod="openshift-ovn-kubernetes/ovnkube-node-mn9vl"
Oct 30 21:54:54.583437 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.583245    6133 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/5d5057fc-dd8a-4dce-b8bb-70b2dc3b074c-hostroot\") pod \"multus-jkdxt\" (UID: \"5d5057fc-dd8a-4dce-b8bb-70b2dc3b074c\") " pod="openshift-multus/multus-jkdxt"
Oct 30 21:54:54.583437 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.583254    6133 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/bebdb584-1a96-453d-8a6d-b332e47bec3d-etc-sysconfig\") pod \"tuned-h8csb\" (UID: \"bebdb584-1a96-453d-8a6d-b332e47bec3d\") " pod="openshift-cluster-node-tuning-operator/tuned-h8csb"
Oct 30 21:54:54.583437 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.583264    6133 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5d5057fc-dd8a-4dce-b8bb-70b2dc3b074c-host-var-lib-kubelet\") pod \"multus-jkdxt\" (UID: \"5d5057fc-dd8a-4dce-b8bb-70b2dc3b074c\") " pod="openshift-multus/multus-jkdxt"
Oct 30 21:54:54.583437 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.583281    6133 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/446beacf-70a0-4d47-96ea-bbe1ca4a5f28-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-mn9vl\" (UID: \"446beacf-70a0-4d47-96ea-bbe1ca4a5f28\") " pod="openshift-ovn-kubernetes/ovnkube-node-mn9vl"
Oct 30 21:54:54.583437 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.583288    6133 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/3e60fceb-8682-45cc-a472-f3d7017c3395-cnibin\") pod \"multus-additional-cni-plugins-c9tnl\" (UID: \"3e60fceb-8682-45cc-a472-f3d7017c3395\") " pod="openshift-multus/multus-additional-cni-plugins-c9tnl"
Oct 30 21:54:54.583437 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.583305    6133 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/613f514c-6011-4936-8df5-a59b960cd3a6-host-slash\") pod \"iptables-alerter-kghfh\" (UID: \"613f514c-6011-4936-8df5-a59b960cd3a6\") " pod="openshift-network-operator/iptables-alerter-kghfh"
Oct 30 21:54:54.583437 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.583320    6133 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/446beacf-70a0-4d47-96ea-bbe1ca4a5f28-host-run-netns\") pod \"ovnkube-node-mn9vl\" (UID: \"446beacf-70a0-4d47-96ea-bbe1ca4a5f28\") " pod="openshift-ovn-kubernetes/ovnkube-node-mn9vl"
Oct 30 21:54:54.583437 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.583330    6133 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5d5057fc-dd8a-4dce-b8bb-70b2dc3b074c-host-var-lib-kubelet\") pod \"multus-jkdxt\" (UID: \"5d5057fc-dd8a-4dce-b8bb-70b2dc3b074c\") " pod="openshift-multus/multus-jkdxt"
Oct 30 21:54:54.583437 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.583334    6133 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/446beacf-70a0-4d47-96ea-bbe1ca4a5f28-run-openvswitch\") pod \"ovnkube-node-mn9vl\" (UID: \"446beacf-70a0-4d47-96ea-bbe1ca4a5f28\") " pod="openshift-ovn-kubernetes/ovnkube-node-mn9vl"
Oct 30 21:54:54.583437 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.583346    6133 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/3e60fceb-8682-45cc-a472-f3d7017c3395-cnibin\") pod \"multus-additional-cni-plugins-c9tnl\" (UID: \"3e60fceb-8682-45cc-a472-f3d7017c3395\") " pod="openshift-multus/multus-additional-cni-plugins-c9tnl"
Oct 30 21:54:54.583437 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.583356    6133 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/446beacf-70a0-4d47-96ea-bbe1ca4a5f28-host-run-netns\") pod \"ovnkube-node-mn9vl\" (UID: \"446beacf-70a0-4d47-96ea-bbe1ca4a5f28\") " pod="openshift-ovn-kubernetes/ovnkube-node-mn9vl"
Oct 30 21:54:54.583437 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.583367    6133 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/446beacf-70a0-4d47-96ea-bbe1ca4a5f28-run-openvswitch\") pod \"ovnkube-node-mn9vl\" (UID: \"446beacf-70a0-4d47-96ea-bbe1ca4a5f28\") " pod="openshift-ovn-kubernetes/ovnkube-node-mn9vl"
Oct 30 21:54:54.583437 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.583373    6133 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/613f514c-6011-4936-8df5-a59b960cd3a6-host-slash\") pod \"iptables-alerter-kghfh\" (UID: \"613f514c-6011-4936-8df5-a59b960cd3a6\") " pod="openshift-network-operator/iptables-alerter-kghfh"
Oct 30 21:54:54.583437 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.583400    6133 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5d5057fc-dd8a-4dce-b8bb-70b2dc3b074c-etc-kubernetes\") pod \"multus-jkdxt\" (UID: \"5d5057fc-dd8a-4dce-b8bb-70b2dc3b074c\") " pod="openshift-multus/multus-jkdxt"
Oct 30 21:54:54.583437 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.583424    6133 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/7ddfa67d-20a0-473d-937f-25fd8e0ba80e-root\") pod \"node-exporter-df5jt\" (UID: \"7ddfa67d-20a0-473d-937f-25fd8e0ba80e\") " pod="openshift-monitoring/node-exporter-df5jt"
Oct 30 21:54:54.583437 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.583451    6133 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5d5057fc-dd8a-4dce-b8bb-70b2dc3b074c-etc-kubernetes\") pod \"multus-jkdxt\" (UID: \"5d5057fc-dd8a-4dce-b8bb-70b2dc3b074c\") " pod="openshift-multus/multus-jkdxt"
Oct 30 21:54:54.583437 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:54.583485    6133 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/7ddfa67d-20a0-473d-937f-25fd8e0ba80e-root\") pod \"node-exporter-df5jt\" (UID: \"7ddfa67d-20a0-473d-937f-25fd8e0ba80e\") " pod="openshift-monitoring/node-exporter-df5jt"
Oct 30 21:54:55.263111 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:55.263076    6133 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:54:55.770846 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:55.770818    6133 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-mn9vl"
Oct 30 21:54:55.863498 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:55.863469    6133 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-868d6f78df-nx2k6"
Oct 30 21:54:56.622328 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:54:56.622296    6133 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-fqmbc"
Oct 30 21:55:00.774578 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:00.774545    6133 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-77f4f76594-htsvq"
Oct 30 21:55:02.949708 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:02.949675    6133 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-mn9vl"
Oct 30 21:55:02.950105 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:02.949889    6133 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Oct 30 21:55:03.611728 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:03.611690    6133 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-fvbz8"
Oct 30 21:55:08.206804 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:08.206729    6133 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-mn9vl"
Oct 30 21:55:09.525954 ip-10-0-143-246 kubenswrapper[6133]: E1030 21:55:09.525916    6133 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6jppj" containerName="extractor"
Oct 30 21:55:11.092209 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:11.092171    6133 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Oct 30 21:55:11.092734 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:11.092687    6133 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="bd93518f-2f96-4483-9e92-513109dd37cb" containerName="prometheus" containerID="cri-o://fc4e2bd2f898fb51f973cbd22a4c415acb3e71bb1ccc34e58f5925bda9c48eda" gracePeriod=600
Oct 30 21:55:11.092828 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:11.092772    6133 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="bd93518f-2f96-4483-9e92-513109dd37cb" containerName="thanos-sidecar" containerID="cri-o://ac3251b59fa7c86209ea54ce01461512a94cf3ecfb802abd04895d4e0b226da3" gracePeriod=600
Oct 30 21:55:11.092828 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:11.092781    6133 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="bd93518f-2f96-4483-9e92-513109dd37cb" containerName="kube-rbac-proxy-thanos" containerID="cri-o://9915681fb0e03c6fd0b8260c96d61e8c11ce6f9d448cf02897ca8c9db3796828" gracePeriod=600
Oct 30 21:55:11.092921 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:11.092803    6133 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="bd93518f-2f96-4483-9e92-513109dd37cb" containerName="config-reloader" containerID="cri-o://7f5d14c4283a270b49a30f65cd969f9059aa7d43f0f4a78590276fa736dae993" gracePeriod=600
Oct 30 21:55:11.092921 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:11.092809    6133 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="bd93518f-2f96-4483-9e92-513109dd37cb" containerName="kube-rbac-proxy-web" containerID="cri-o://d76784ca45eef8daaa0b9110107077e348d4d174313732f2cc153bbbff9d32eb" gracePeriod=600
Oct 30 21:55:11.092921 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:11.092746    6133 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="bd93518f-2f96-4483-9e92-513109dd37cb" containerName="kube-rbac-proxy" containerID="cri-o://4b5654280a0b3c41ca38a375f20091c72976104b1839d8f9f2405f57b1570677" gracePeriod=600
Oct 30 21:55:11.362049 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:11.361960    6133 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:55:11.482182 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:11.482151    6133 generic.go:356] "Generic (PLEG): container finished" podID="bd93518f-2f96-4483-9e92-513109dd37cb" containerID="9915681fb0e03c6fd0b8260c96d61e8c11ce6f9d448cf02897ca8c9db3796828" exitCode=0
Oct 30 21:55:11.482182 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:11.482174    6133 generic.go:356] "Generic (PLEG): container finished" podID="bd93518f-2f96-4483-9e92-513109dd37cb" containerID="4b5654280a0b3c41ca38a375f20091c72976104b1839d8f9f2405f57b1570677" exitCode=0
Oct 30 21:55:11.482182 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:11.482182    6133 generic.go:356] "Generic (PLEG): container finished" podID="bd93518f-2f96-4483-9e92-513109dd37cb" containerID="d76784ca45eef8daaa0b9110107077e348d4d174313732f2cc153bbbff9d32eb" exitCode=0
Oct 30 21:55:11.482182 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:11.482190    6133 generic.go:356] "Generic (PLEG): container finished" podID="bd93518f-2f96-4483-9e92-513109dd37cb" containerID="ac3251b59fa7c86209ea54ce01461512a94cf3ecfb802abd04895d4e0b226da3" exitCode=0
Oct 30 21:55:11.482450 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:11.482197    6133 generic.go:356] "Generic (PLEG): container finished" podID="bd93518f-2f96-4483-9e92-513109dd37cb" containerID="7f5d14c4283a270b49a30f65cd969f9059aa7d43f0f4a78590276fa736dae993" exitCode=0
Oct 30 21:55:11.482450 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:11.482202    6133 generic.go:356] "Generic (PLEG): container finished" podID="bd93518f-2f96-4483-9e92-513109dd37cb" containerID="fc4e2bd2f898fb51f973cbd22a4c415acb3e71bb1ccc34e58f5925bda9c48eda" exitCode=0
Oct 30 21:55:11.482450 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:11.482232    6133 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:55:11.482450 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:11.482231    6133 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"bd93518f-2f96-4483-9e92-513109dd37cb","Type":"ContainerDied","Data":"9915681fb0e03c6fd0b8260c96d61e8c11ce6f9d448cf02897ca8c9db3796828"}
Oct 30 21:55:11.482450 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:11.482303    6133 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"bd93518f-2f96-4483-9e92-513109dd37cb","Type":"ContainerDied","Data":"4b5654280a0b3c41ca38a375f20091c72976104b1839d8f9f2405f57b1570677"}
Oct 30 21:55:11.482450 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:11.482332    6133 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"bd93518f-2f96-4483-9e92-513109dd37cb","Type":"ContainerDied","Data":"d76784ca45eef8daaa0b9110107077e348d4d174313732f2cc153bbbff9d32eb"}
Oct 30 21:55:11.482450 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:11.482356    6133 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"bd93518f-2f96-4483-9e92-513109dd37cb","Type":"ContainerDied","Data":"ac3251b59fa7c86209ea54ce01461512a94cf3ecfb802abd04895d4e0b226da3"}
Oct 30 21:55:11.482450 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:11.482369    6133 scope.go:122] "RemoveContainer" containerID="9915681fb0e03c6fd0b8260c96d61e8c11ce6f9d448cf02897ca8c9db3796828"
Oct 30 21:55:11.482450 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:11.482379    6133 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"bd93518f-2f96-4483-9e92-513109dd37cb","Type":"ContainerDied","Data":"7f5d14c4283a270b49a30f65cd969f9059aa7d43f0f4a78590276fa736dae993"}
Oct 30 21:55:11.482450 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:11.482404    6133 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"bd93518f-2f96-4483-9e92-513109dd37cb","Type":"ContainerDied","Data":"fc4e2bd2f898fb51f973cbd22a4c415acb3e71bb1ccc34e58f5925bda9c48eda"}
Oct 30 21:55:11.482450 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:11.482428    6133 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"bd93518f-2f96-4483-9e92-513109dd37cb","Type":"ContainerDied","Data":"83f5c0d6735a2e35b4e005074f2c7f583f92ea7a94d680c9c430b8d1df568d4c"}
Oct 30 21:55:11.489942 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:11.489926    6133 scope.go:122] "RemoveContainer" containerID="4b5654280a0b3c41ca38a375f20091c72976104b1839d8f9f2405f57b1570677"
Oct 30 21:55:11.496817 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:11.496798    6133 scope.go:122] "RemoveContainer" containerID="d76784ca45eef8daaa0b9110107077e348d4d174313732f2cc153bbbff9d32eb"
Oct 30 21:55:11.503702 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:11.503685    6133 scope.go:122] "RemoveContainer" containerID="ac3251b59fa7c86209ea54ce01461512a94cf3ecfb802abd04895d4e0b226da3"
Oct 30 21:55:11.510456 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:11.510438    6133 scope.go:122] "RemoveContainer" containerID="7f5d14c4283a270b49a30f65cd969f9059aa7d43f0f4a78590276fa736dae993"
Oct 30 21:55:11.517567 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:11.517549    6133 scope.go:122] "RemoveContainer" containerID="fc4e2bd2f898fb51f973cbd22a4c415acb3e71bb1ccc34e58f5925bda9c48eda"
Oct 30 21:55:11.521320 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:11.521298    6133 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/bd93518f-2f96-4483-9e92-513109dd37cb-secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/bd93518f-2f96-4483-9e92-513109dd37cb-secret-grpc-tls\") pod \"bd93518f-2f96-4483-9e92-513109dd37cb\" (UID: \"bd93518f-2f96-4483-9e92-513109dd37cb\") "
Oct 30 21:55:11.521427 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:11.521337    6133 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/bd93518f-2f96-4483-9e92-513109dd37cb-configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bd93518f-2f96-4483-9e92-513109dd37cb-configmap-serving-certs-ca-bundle\") pod \"bd93518f-2f96-4483-9e92-513109dd37cb\" (UID: \"bd93518f-2f96-4483-9e92-513109dd37cb\") "
Oct 30 21:55:11.521427 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:11.521373    6133 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/bd93518f-2f96-4483-9e92-513109dd37cb-prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bd93518f-2f96-4483-9e92-513109dd37cb-prometheus-trusted-ca-bundle\") pod \"bd93518f-2f96-4483-9e92-513109dd37cb\" (UID: \"bd93518f-2f96-4483-9e92-513109dd37cb\") "
Oct 30 21:55:11.521540 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:11.521483    6133 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/bd93518f-2f96-4483-9e92-513109dd37cb-configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bd93518f-2f96-4483-9e92-513109dd37cb-configmap-kubelet-serving-ca-bundle\") pod \"bd93518f-2f96-4483-9e92-513109dd37cb\" (UID: \"bd93518f-2f96-4483-9e92-513109dd37cb\") "
Oct 30 21:55:11.521540 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:11.521531    6133 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/bd93518f-2f96-4483-9e92-513109dd37cb-tls-assets\" (UniqueName: \"kubernetes.io/projected/bd93518f-2f96-4483-9e92-513109dd37cb-tls-assets\") pod \"bd93518f-2f96-4483-9e92-513109dd37cb\" (UID: \"bd93518f-2f96-4483-9e92-513109dd37cb\") "
Oct 30 21:55:11.521681 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:11.521566    6133 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/bd93518f-2f96-4483-9e92-513109dd37cb-kube-api-access-qrt69\" (UniqueName: \"kubernetes.io/projected/bd93518f-2f96-4483-9e92-513109dd37cb-kube-api-access-qrt69\") pod \"bd93518f-2f96-4483-9e92-513109dd37cb\" (UID: \"bd93518f-2f96-4483-9e92-513109dd37cb\") "
Oct 30 21:55:11.521681 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:11.521605    6133 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bd93518f-2f96-4483-9e92-513109dd37cb-config-out\" (UniqueName: \"kubernetes.io/empty-dir/bd93518f-2f96-4483-9e92-513109dd37cb-config-out\") pod \"bd93518f-2f96-4483-9e92-513109dd37cb\" (UID: \"bd93518f-2f96-4483-9e92-513109dd37cb\") "
Oct 30 21:55:11.521681 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:11.521646    6133 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/bd93518f-2f96-4483-9e92-513109dd37cb-prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/bd93518f-2f96-4483-9e92-513109dd37cb-prometheus-k8s-rulefiles-0\") pod \"bd93518f-2f96-4483-9e92-513109dd37cb\" (UID: \"bd93518f-2f96-4483-9e92-513109dd37cb\") "
Oct 30 21:55:11.521681 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:11.521674    6133 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/bd93518f-2f96-4483-9e92-513109dd37cb-web-config\" (UniqueName: \"kubernetes.io/secret/bd93518f-2f96-4483-9e92-513109dd37cb-web-config\") pod \"bd93518f-2f96-4483-9e92-513109dd37cb\" (UID: \"bd93518f-2f96-4483-9e92-513109dd37cb\") "
Oct 30 21:55:11.521874 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:11.521719    6133 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/bd93518f-2f96-4483-9e92-513109dd37cb-secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/bd93518f-2f96-4483-9e92-513109dd37cb-secret-kube-rbac-proxy\") pod \"bd93518f-2f96-4483-9e92-513109dd37cb\" (UID: \"bd93518f-2f96-4483-9e92-513109dd37cb\") "
Oct 30 21:55:11.521874 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:11.521740    6133 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bd93518f-2f96-4483-9e92-513109dd37cb-configmap-serving-certs-ca-bundle" pod "bd93518f-2f96-4483-9e92-513109dd37cb" (UID: "bd93518f-2f96-4483-9e92-513109dd37cb"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:55:11.521874 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:11.521758    6133 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/bd93518f-2f96-4483-9e92-513109dd37cb-secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/bd93518f-2f96-4483-9e92-513109dd37cb-secret-prometheus-k8s-tls\") pod \"bd93518f-2f96-4483-9e92-513109dd37cb\" (UID: \"bd93518f-2f96-4483-9e92-513109dd37cb\") "
Oct 30 21:55:11.521874 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:11.521791    6133 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/bd93518f-2f96-4483-9e92-513109dd37cb-secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/bd93518f-2f96-4483-9e92-513109dd37cb-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"bd93518f-2f96-4483-9e92-513109dd37cb\" (UID: \"bd93518f-2f96-4483-9e92-513109dd37cb\") "
Oct 30 21:55:11.522156 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:11.522135    6133 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/bd93518f-2f96-4483-9e92-513109dd37cb-secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/bd93518f-2f96-4483-9e92-513109dd37cb-secret-metrics-client-certs\") pod \"bd93518f-2f96-4483-9e92-513109dd37cb\" (UID: \"bd93518f-2f96-4483-9e92-513109dd37cb\") "
Oct 30 21:55:11.522227 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:11.522209    6133 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/bd93518f-2f96-4483-9e92-513109dd37cb-secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/bd93518f-2f96-4483-9e92-513109dd37cb-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"bd93518f-2f96-4483-9e92-513109dd37cb\" (UID: \"bd93518f-2f96-4483-9e92-513109dd37cb\") "
Oct 30 21:55:11.522280 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:11.522259    6133 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bd93518f-2f96-4483-9e92-513109dd37cb-prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/bd93518f-2f96-4483-9e92-513109dd37cb-prometheus-k8s-db\") pod \"bd93518f-2f96-4483-9e92-513109dd37cb\" (UID: \"bd93518f-2f96-4483-9e92-513109dd37cb\") "
Oct 30 21:55:11.522332 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:11.522308    6133 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/bd93518f-2f96-4483-9e92-513109dd37cb-thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/bd93518f-2f96-4483-9e92-513109dd37cb-thanos-prometheus-http-client-file\") pod \"bd93518f-2f96-4483-9e92-513109dd37cb\" (UID: \"bd93518f-2f96-4483-9e92-513109dd37cb\") "
Oct 30 21:55:11.522382 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:11.522337    6133 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/bd93518f-2f96-4483-9e92-513109dd37cb-config\" (UniqueName: \"kubernetes.io/secret/bd93518f-2f96-4483-9e92-513109dd37cb-config\") pod \"bd93518f-2f96-4483-9e92-513109dd37cb\" (UID: \"bd93518f-2f96-4483-9e92-513109dd37cb\") "
Oct 30 21:55:11.522382 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:11.522371    6133 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/bd93518f-2f96-4483-9e92-513109dd37cb-configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bd93518f-2f96-4483-9e92-513109dd37cb-configmap-metrics-client-ca\") pod \"bd93518f-2f96-4483-9e92-513109dd37cb\" (UID: \"bd93518f-2f96-4483-9e92-513109dd37cb\") "
Oct 30 21:55:11.522484 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:11.522408    6133 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/bd93518f-2f96-4483-9e92-513109dd37cb-prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/bd93518f-2f96-4483-9e92-513109dd37cb-prometheus-k8s-rulefiles-2\") pod \"bd93518f-2f96-4483-9e92-513109dd37cb\" (UID: \"bd93518f-2f96-4483-9e92-513109dd37cb\") "
Oct 30 21:55:11.522484 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:11.522444    6133 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/bd93518f-2f96-4483-9e92-513109dd37cb-prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/bd93518f-2f96-4483-9e92-513109dd37cb-prometheus-k8s-rulefiles-1\") pod \"bd93518f-2f96-4483-9e92-513109dd37cb\" (UID: \"bd93518f-2f96-4483-9e92-513109dd37cb\") "
Oct 30 21:55:11.522584 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:11.522477    6133 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bd93518f-2f96-4483-9e92-513109dd37cb-configmap-kubelet-serving-ca-bundle" pod "bd93518f-2f96-4483-9e92-513109dd37cb" (UID: "bd93518f-2f96-4483-9e92-513109dd37cb"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:55:11.523041 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:11.523014    6133 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bd93518f-2f96-4483-9e92-513109dd37cb-prometheus-trusted-ca-bundle" pod "bd93518f-2f96-4483-9e92-513109dd37cb" (UID: "bd93518f-2f96-4483-9e92-513109dd37cb"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:55:11.523602 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:11.523541    6133 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bd93518f-2f96-4483-9e92-513109dd37cb-configmap-metrics-client-ca" pod "bd93518f-2f96-4483-9e92-513109dd37cb" (UID: "bd93518f-2f96-4483-9e92-513109dd37cb"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:55:11.525301 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:11.524878    6133 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bd93518f-2f96-4483-9e92-513109dd37cb-prometheus-k8s-rulefiles-2" pod "bd93518f-2f96-4483-9e92-513109dd37cb" (UID: "bd93518f-2f96-4483-9e92-513109dd37cb"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:55:11.525843 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:11.525817    6133 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bd93518f-2f96-4483-9e92-513109dd37cb-secret-grpc-tls" pod "bd93518f-2f96-4483-9e92-513109dd37cb" (UID: "bd93518f-2f96-4483-9e92-513109dd37cb"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:55:11.526226 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:11.526197    6133 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bd93518f-2f96-4483-9e92-513109dd37cb-prometheus-k8s-db" pod "bd93518f-2f96-4483-9e92-513109dd37cb" (UID: "bd93518f-2f96-4483-9e92-513109dd37cb"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:55:11.527559 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:11.527429    6133 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/bd93518f-2f96-4483-9e92-513109dd37cb-prometheus-k8s-db\") on node \"ip-10-0-143-246.ec2.internal\" DevicePath \"\""
Oct 30 21:55:11.527559 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:11.527468    6133 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bd93518f-2f96-4483-9e92-513109dd37cb-configmap-metrics-client-ca\") on node \"ip-10-0-143-246.ec2.internal\" DevicePath \"\""
Oct 30 21:55:11.527559 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:11.527485    6133 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/bd93518f-2f96-4483-9e92-513109dd37cb-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-143-246.ec2.internal\" DevicePath \"\""
Oct 30 21:55:11.527559 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:11.527500    6133 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/bd93518f-2f96-4483-9e92-513109dd37cb-secret-grpc-tls\") on node \"ip-10-0-143-246.ec2.internal\" DevicePath \"\""
Oct 30 21:55:11.527559 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:11.527515    6133 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bd93518f-2f96-4483-9e92-513109dd37cb-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-143-246.ec2.internal\" DevicePath \"\""
Oct 30 21:55:11.527559 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:11.527536    6133 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bd93518f-2f96-4483-9e92-513109dd37cb-prometheus-trusted-ca-bundle\") on node \"ip-10-0-143-246.ec2.internal\" DevicePath \"\""
Oct 30 21:55:11.527559 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:11.527550    6133 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bd93518f-2f96-4483-9e92-513109dd37cb-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-143-246.ec2.internal\" DevicePath \"\""
Oct 30 21:55:11.527905 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:11.527665    6133 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bd93518f-2f96-4483-9e92-513109dd37cb-prometheus-k8s-rulefiles-1" pod "bd93518f-2f96-4483-9e92-513109dd37cb" (UID: "bd93518f-2f96-4483-9e92-513109dd37cb"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:55:11.527905 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:11.527865    6133 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bd93518f-2f96-4483-9e92-513109dd37cb-secret-prometheus-k8s-tls" pod "bd93518f-2f96-4483-9e92-513109dd37cb" (UID: "bd93518f-2f96-4483-9e92-513109dd37cb"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:55:11.528147 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:11.528121    6133 scope.go:122] "RemoveContainer" containerID="1afbd58205aa89f01dece6ea4aceecdb2c1a95d6f90583bfa5c329026ead89df"
Oct 30 21:55:11.528204 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:11.528176    6133 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bd93518f-2f96-4483-9e92-513109dd37cb-secret-prometheus-k8s-thanos-sidecar-tls" pod "bd93518f-2f96-4483-9e92-513109dd37cb" (UID: "bd93518f-2f96-4483-9e92-513109dd37cb"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:55:11.528896 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:11.528866    6133 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bd93518f-2f96-4483-9e92-513109dd37cb-prometheus-k8s-rulefiles-0" pod "bd93518f-2f96-4483-9e92-513109dd37cb" (UID: "bd93518f-2f96-4483-9e92-513109dd37cb"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:55:11.531053 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:11.529981    6133 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bd93518f-2f96-4483-9e92-513109dd37cb-config" pod "bd93518f-2f96-4483-9e92-513109dd37cb" (UID: "bd93518f-2f96-4483-9e92-513109dd37cb"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:55:11.531053 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:11.530577    6133 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bd93518f-2f96-4483-9e92-513109dd37cb-secret-metrics-client-certs" pod "bd93518f-2f96-4483-9e92-513109dd37cb" (UID: "bd93518f-2f96-4483-9e92-513109dd37cb"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:55:11.533041 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:11.531786    6133 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bd93518f-2f96-4483-9e92-513109dd37cb-config-out" pod "bd93518f-2f96-4483-9e92-513109dd37cb" (UID: "bd93518f-2f96-4483-9e92-513109dd37cb"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:55:11.533041 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:11.531865    6133 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bd93518f-2f96-4483-9e92-513109dd37cb-kube-api-access-qrt69" pod "bd93518f-2f96-4483-9e92-513109dd37cb" (UID: "bd93518f-2f96-4483-9e92-513109dd37cb"). InnerVolumeSpecName "kube-api-access-qrt69". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:55:11.533041 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:11.531902    6133 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bd93518f-2f96-4483-9e92-513109dd37cb-tls-assets" pod "bd93518f-2f96-4483-9e92-513109dd37cb" (UID: "bd93518f-2f96-4483-9e92-513109dd37cb"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:55:11.533041 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:11.531943    6133 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bd93518f-2f96-4483-9e92-513109dd37cb-secret-kube-rbac-proxy" pod "bd93518f-2f96-4483-9e92-513109dd37cb" (UID: "bd93518f-2f96-4483-9e92-513109dd37cb"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:55:11.533041 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:11.532640    6133 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bd93518f-2f96-4483-9e92-513109dd37cb-thanos-prometheus-http-client-file" pod "bd93518f-2f96-4483-9e92-513109dd37cb" (UID: "bd93518f-2f96-4483-9e92-513109dd37cb"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:55:11.533289 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:11.533064    6133 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bd93518f-2f96-4483-9e92-513109dd37cb-secret-prometheus-k8s-kube-rbac-proxy-web" pod "bd93518f-2f96-4483-9e92-513109dd37cb" (UID: "bd93518f-2f96-4483-9e92-513109dd37cb"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:55:11.537732 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:11.537712    6133 scope.go:122] "RemoveContainer" containerID="9915681fb0e03c6fd0b8260c96d61e8c11ce6f9d448cf02897ca8c9db3796828"
Oct 30 21:55:11.538160 ip-10-0-143-246 kubenswrapper[6133]: E1030 21:55:11.537980    6133 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9915681fb0e03c6fd0b8260c96d61e8c11ce6f9d448cf02897ca8c9db3796828\": container with ID starting with 9915681fb0e03c6fd0b8260c96d61e8c11ce6f9d448cf02897ca8c9db3796828 not found: ID does not exist" containerID="9915681fb0e03c6fd0b8260c96d61e8c11ce6f9d448cf02897ca8c9db3796828"
Oct 30 21:55:11.538236 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:11.538171    6133 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9915681fb0e03c6fd0b8260c96d61e8c11ce6f9d448cf02897ca8c9db3796828"} err="failed to get container status \"9915681fb0e03c6fd0b8260c96d61e8c11ce6f9d448cf02897ca8c9db3796828\": rpc error: code = NotFound desc = could not find container \"9915681fb0e03c6fd0b8260c96d61e8c11ce6f9d448cf02897ca8c9db3796828\": container with ID starting with 9915681fb0e03c6fd0b8260c96d61e8c11ce6f9d448cf02897ca8c9db3796828 not found: ID does not exist"
Oct 30 21:55:11.538236 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:11.538209    6133 scope.go:122] "RemoveContainer" containerID="4b5654280a0b3c41ca38a375f20091c72976104b1839d8f9f2405f57b1570677"
Oct 30 21:55:11.538496 ip-10-0-143-246 kubenswrapper[6133]: E1030 21:55:11.538479    6133 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4b5654280a0b3c41ca38a375f20091c72976104b1839d8f9f2405f57b1570677\": container with ID starting with 4b5654280a0b3c41ca38a375f20091c72976104b1839d8f9f2405f57b1570677 not found: ID does not exist" containerID="4b5654280a0b3c41ca38a375f20091c72976104b1839d8f9f2405f57b1570677"
Oct 30 21:55:11.538551 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:11.538501    6133 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4b5654280a0b3c41ca38a375f20091c72976104b1839d8f9f2405f57b1570677"} err="failed to get container status \"4b5654280a0b3c41ca38a375f20091c72976104b1839d8f9f2405f57b1570677\": rpc error: code = NotFound desc = could not find container \"4b5654280a0b3c41ca38a375f20091c72976104b1839d8f9f2405f57b1570677\": container with ID starting with 4b5654280a0b3c41ca38a375f20091c72976104b1839d8f9f2405f57b1570677 not found: ID does not exist"
Oct 30 21:55:11.538551 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:11.538516    6133 scope.go:122] "RemoveContainer" containerID="d76784ca45eef8daaa0b9110107077e348d4d174313732f2cc153bbbff9d32eb"
Oct 30 21:55:11.538789 ip-10-0-143-246 kubenswrapper[6133]: E1030 21:55:11.538768    6133 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d76784ca45eef8daaa0b9110107077e348d4d174313732f2cc153bbbff9d32eb\": container with ID starting with d76784ca45eef8daaa0b9110107077e348d4d174313732f2cc153bbbff9d32eb not found: ID does not exist" containerID="d76784ca45eef8daaa0b9110107077e348d4d174313732f2cc153bbbff9d32eb"
Oct 30 21:55:11.538853 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:11.538798    6133 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d76784ca45eef8daaa0b9110107077e348d4d174313732f2cc153bbbff9d32eb"} err="failed to get container status \"d76784ca45eef8daaa0b9110107077e348d4d174313732f2cc153bbbff9d32eb\": rpc error: code = NotFound desc = could not find container \"d76784ca45eef8daaa0b9110107077e348d4d174313732f2cc153bbbff9d32eb\": container with ID starting with d76784ca45eef8daaa0b9110107077e348d4d174313732f2cc153bbbff9d32eb not found: ID does not exist"
Oct 30 21:55:11.538853 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:11.538814    6133 scope.go:122] "RemoveContainer" containerID="ac3251b59fa7c86209ea54ce01461512a94cf3ecfb802abd04895d4e0b226da3"
Oct 30 21:55:11.539083 ip-10-0-143-246 kubenswrapper[6133]: E1030 21:55:11.539058    6133 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ac3251b59fa7c86209ea54ce01461512a94cf3ecfb802abd04895d4e0b226da3\": container with ID starting with ac3251b59fa7c86209ea54ce01461512a94cf3ecfb802abd04895d4e0b226da3 not found: ID does not exist" containerID="ac3251b59fa7c86209ea54ce01461512a94cf3ecfb802abd04895d4e0b226da3"
Oct 30 21:55:11.539134 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:11.539088    6133 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ac3251b59fa7c86209ea54ce01461512a94cf3ecfb802abd04895d4e0b226da3"} err="failed to get container status \"ac3251b59fa7c86209ea54ce01461512a94cf3ecfb802abd04895d4e0b226da3\": rpc error: code = NotFound desc = could not find container \"ac3251b59fa7c86209ea54ce01461512a94cf3ecfb802abd04895d4e0b226da3\": container with ID starting with ac3251b59fa7c86209ea54ce01461512a94cf3ecfb802abd04895d4e0b226da3 not found: ID does not exist"
Oct 30 21:55:11.539134 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:11.539102    6133 scope.go:122] "RemoveContainer" containerID="7f5d14c4283a270b49a30f65cd969f9059aa7d43f0f4a78590276fa736dae993"
Oct 30 21:55:11.539347 ip-10-0-143-246 kubenswrapper[6133]: E1030 21:55:11.539333    6133 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7f5d14c4283a270b49a30f65cd969f9059aa7d43f0f4a78590276fa736dae993\": container with ID starting with 7f5d14c4283a270b49a30f65cd969f9059aa7d43f0f4a78590276fa736dae993 not found: ID does not exist" containerID="7f5d14c4283a270b49a30f65cd969f9059aa7d43f0f4a78590276fa736dae993"
Oct 30 21:55:11.539389 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:11.539350    6133 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7f5d14c4283a270b49a30f65cd969f9059aa7d43f0f4a78590276fa736dae993"} err="failed to get container status \"7f5d14c4283a270b49a30f65cd969f9059aa7d43f0f4a78590276fa736dae993\": rpc error: code = NotFound desc = could not find container \"7f5d14c4283a270b49a30f65cd969f9059aa7d43f0f4a78590276fa736dae993\": container with ID starting with 7f5d14c4283a270b49a30f65cd969f9059aa7d43f0f4a78590276fa736dae993 not found: ID does not exist"
Oct 30 21:55:11.539389 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:11.539362    6133 scope.go:122] "RemoveContainer" containerID="fc4e2bd2f898fb51f973cbd22a4c415acb3e71bb1ccc34e58f5925bda9c48eda"
Oct 30 21:55:11.539579 ip-10-0-143-246 kubenswrapper[6133]: E1030 21:55:11.539560    6133 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fc4e2bd2f898fb51f973cbd22a4c415acb3e71bb1ccc34e58f5925bda9c48eda\": container with ID starting with fc4e2bd2f898fb51f973cbd22a4c415acb3e71bb1ccc34e58f5925bda9c48eda not found: ID does not exist" containerID="fc4e2bd2f898fb51f973cbd22a4c415acb3e71bb1ccc34e58f5925bda9c48eda"
Oct 30 21:55:11.539618 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:11.539584    6133 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fc4e2bd2f898fb51f973cbd22a4c415acb3e71bb1ccc34e58f5925bda9c48eda"} err="failed to get container status \"fc4e2bd2f898fb51f973cbd22a4c415acb3e71bb1ccc34e58f5925bda9c48eda\": rpc error: code = NotFound desc = could not find container \"fc4e2bd2f898fb51f973cbd22a4c415acb3e71bb1ccc34e58f5925bda9c48eda\": container with ID starting with fc4e2bd2f898fb51f973cbd22a4c415acb3e71bb1ccc34e58f5925bda9c48eda not found: ID does not exist"
Oct 30 21:55:11.539618 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:11.539599    6133 scope.go:122] "RemoveContainer" containerID="1afbd58205aa89f01dece6ea4aceecdb2c1a95d6f90583bfa5c329026ead89df"
Oct 30 21:55:11.539853 ip-10-0-143-246 kubenswrapper[6133]: E1030 21:55:11.539837    6133 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1afbd58205aa89f01dece6ea4aceecdb2c1a95d6f90583bfa5c329026ead89df\": container with ID starting with 1afbd58205aa89f01dece6ea4aceecdb2c1a95d6f90583bfa5c329026ead89df not found: ID does not exist" containerID="1afbd58205aa89f01dece6ea4aceecdb2c1a95d6f90583bfa5c329026ead89df"
Oct 30 21:55:11.539908 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:11.539855    6133 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1afbd58205aa89f01dece6ea4aceecdb2c1a95d6f90583bfa5c329026ead89df"} err="failed to get container status \"1afbd58205aa89f01dece6ea4aceecdb2c1a95d6f90583bfa5c329026ead89df\": rpc error: code = NotFound desc = could not find container \"1afbd58205aa89f01dece6ea4aceecdb2c1a95d6f90583bfa5c329026ead89df\": container with ID starting with 1afbd58205aa89f01dece6ea4aceecdb2c1a95d6f90583bfa5c329026ead89df not found: ID does not exist"
Oct 30 21:55:11.539908 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:11.539868    6133 scope.go:122] "RemoveContainer" containerID="9915681fb0e03c6fd0b8260c96d61e8c11ce6f9d448cf02897ca8c9db3796828"
Oct 30 21:55:11.540226 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:11.540201    6133 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9915681fb0e03c6fd0b8260c96d61e8c11ce6f9d448cf02897ca8c9db3796828"} err="failed to get container status \"9915681fb0e03c6fd0b8260c96d61e8c11ce6f9d448cf02897ca8c9db3796828\": rpc error: code = NotFound desc = could not find container \"9915681fb0e03c6fd0b8260c96d61e8c11ce6f9d448cf02897ca8c9db3796828\": container with ID starting with 9915681fb0e03c6fd0b8260c96d61e8c11ce6f9d448cf02897ca8c9db3796828 not found: ID does not exist"
Oct 30 21:55:11.540285 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:11.540228    6133 scope.go:122] "RemoveContainer" containerID="4b5654280a0b3c41ca38a375f20091c72976104b1839d8f9f2405f57b1570677"
Oct 30 21:55:11.540493 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:11.540462    6133 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bd93518f-2f96-4483-9e92-513109dd37cb-web-config" pod "bd93518f-2f96-4483-9e92-513109dd37cb" (UID: "bd93518f-2f96-4483-9e92-513109dd37cb"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:55:11.540588 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:11.540536    6133 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4b5654280a0b3c41ca38a375f20091c72976104b1839d8f9f2405f57b1570677"} err="failed to get container status \"4b5654280a0b3c41ca38a375f20091c72976104b1839d8f9f2405f57b1570677\": rpc error: code = NotFound desc = could not find container \"4b5654280a0b3c41ca38a375f20091c72976104b1839d8f9f2405f57b1570677\": container with ID starting with 4b5654280a0b3c41ca38a375f20091c72976104b1839d8f9f2405f57b1570677 not found: ID does not exist"
Oct 30 21:55:11.540588 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:11.540554    6133 scope.go:122] "RemoveContainer" containerID="d76784ca45eef8daaa0b9110107077e348d4d174313732f2cc153bbbff9d32eb"
Oct 30 21:55:11.540826 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:11.540805    6133 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d76784ca45eef8daaa0b9110107077e348d4d174313732f2cc153bbbff9d32eb"} err="failed to get container status \"d76784ca45eef8daaa0b9110107077e348d4d174313732f2cc153bbbff9d32eb\": rpc error: code = NotFound desc = could not find container \"d76784ca45eef8daaa0b9110107077e348d4d174313732f2cc153bbbff9d32eb\": container with ID starting with d76784ca45eef8daaa0b9110107077e348d4d174313732f2cc153bbbff9d32eb not found: ID does not exist"
Oct 30 21:55:11.540903 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:11.540827    6133 scope.go:122] "RemoveContainer" containerID="ac3251b59fa7c86209ea54ce01461512a94cf3ecfb802abd04895d4e0b226da3"
Oct 30 21:55:11.541086 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:11.541068    6133 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ac3251b59fa7c86209ea54ce01461512a94cf3ecfb802abd04895d4e0b226da3"} err="failed to get container status \"ac3251b59fa7c86209ea54ce01461512a94cf3ecfb802abd04895d4e0b226da3\": rpc error: code = NotFound desc = could not find container \"ac3251b59fa7c86209ea54ce01461512a94cf3ecfb802abd04895d4e0b226da3\": container with ID starting with ac3251b59fa7c86209ea54ce01461512a94cf3ecfb802abd04895d4e0b226da3 not found: ID does not exist"
Oct 30 21:55:11.541158 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:11.541087    6133 scope.go:122] "RemoveContainer" containerID="7f5d14c4283a270b49a30f65cd969f9059aa7d43f0f4a78590276fa736dae993"
Oct 30 21:55:11.541338 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:11.541320    6133 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7f5d14c4283a270b49a30f65cd969f9059aa7d43f0f4a78590276fa736dae993"} err="failed to get container status \"7f5d14c4283a270b49a30f65cd969f9059aa7d43f0f4a78590276fa736dae993\": rpc error: code = NotFound desc = could not find container \"7f5d14c4283a270b49a30f65cd969f9059aa7d43f0f4a78590276fa736dae993\": container with ID starting with 7f5d14c4283a270b49a30f65cd969f9059aa7d43f0f4a78590276fa736dae993 not found: ID does not exist"
Oct 30 21:55:11.541390 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:11.541338    6133 scope.go:122] "RemoveContainer" containerID="fc4e2bd2f898fb51f973cbd22a4c415acb3e71bb1ccc34e58f5925bda9c48eda"
Oct 30 21:55:11.541583 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:11.541565    6133 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fc4e2bd2f898fb51f973cbd22a4c415acb3e71bb1ccc34e58f5925bda9c48eda"} err="failed to get container status \"fc4e2bd2f898fb51f973cbd22a4c415acb3e71bb1ccc34e58f5925bda9c48eda\": rpc error: code = NotFound desc = could not find container \"fc4e2bd2f898fb51f973cbd22a4c415acb3e71bb1ccc34e58f5925bda9c48eda\": container with ID starting with fc4e2bd2f898fb51f973cbd22a4c415acb3e71bb1ccc34e58f5925bda9c48eda not found: ID does not exist"
Oct 30 21:55:11.541653 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:11.541585    6133 scope.go:122] "RemoveContainer" containerID="1afbd58205aa89f01dece6ea4aceecdb2c1a95d6f90583bfa5c329026ead89df"
Oct 30 21:55:11.541810 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:11.541792    6133 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1afbd58205aa89f01dece6ea4aceecdb2c1a95d6f90583bfa5c329026ead89df"} err="failed to get container status \"1afbd58205aa89f01dece6ea4aceecdb2c1a95d6f90583bfa5c329026ead89df\": rpc error: code = NotFound desc = could not find container \"1afbd58205aa89f01dece6ea4aceecdb2c1a95d6f90583bfa5c329026ead89df\": container with ID starting with 1afbd58205aa89f01dece6ea4aceecdb2c1a95d6f90583bfa5c329026ead89df not found: ID does not exist"
Oct 30 21:55:11.541894 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:11.541810    6133 scope.go:122] "RemoveContainer" containerID="9915681fb0e03c6fd0b8260c96d61e8c11ce6f9d448cf02897ca8c9db3796828"
Oct 30 21:55:11.542059 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:11.542042    6133 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9915681fb0e03c6fd0b8260c96d61e8c11ce6f9d448cf02897ca8c9db3796828"} err="failed to get container status \"9915681fb0e03c6fd0b8260c96d61e8c11ce6f9d448cf02897ca8c9db3796828\": rpc error: code = NotFound desc = could not find container \"9915681fb0e03c6fd0b8260c96d61e8c11ce6f9d448cf02897ca8c9db3796828\": container with ID starting with 9915681fb0e03c6fd0b8260c96d61e8c11ce6f9d448cf02897ca8c9db3796828 not found: ID does not exist"
Oct 30 21:55:11.542113 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:11.542059    6133 scope.go:122] "RemoveContainer" containerID="4b5654280a0b3c41ca38a375f20091c72976104b1839d8f9f2405f57b1570677"
Oct 30 21:55:11.542278 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:11.542260    6133 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4b5654280a0b3c41ca38a375f20091c72976104b1839d8f9f2405f57b1570677"} err="failed to get container status \"4b5654280a0b3c41ca38a375f20091c72976104b1839d8f9f2405f57b1570677\": rpc error: code = NotFound desc = could not find container \"4b5654280a0b3c41ca38a375f20091c72976104b1839d8f9f2405f57b1570677\": container with ID starting with 4b5654280a0b3c41ca38a375f20091c72976104b1839d8f9f2405f57b1570677 not found: ID does not exist"
Oct 30 21:55:11.542318 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:11.542279    6133 scope.go:122] "RemoveContainer" containerID="d76784ca45eef8daaa0b9110107077e348d4d174313732f2cc153bbbff9d32eb"
Oct 30 21:55:11.542500 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:11.542482    6133 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d76784ca45eef8daaa0b9110107077e348d4d174313732f2cc153bbbff9d32eb"} err="failed to get container status \"d76784ca45eef8daaa0b9110107077e348d4d174313732f2cc153bbbff9d32eb\": rpc error: code = NotFound desc = could not find container \"d76784ca45eef8daaa0b9110107077e348d4d174313732f2cc153bbbff9d32eb\": container with ID starting with d76784ca45eef8daaa0b9110107077e348d4d174313732f2cc153bbbff9d32eb not found: ID does not exist"
Oct 30 21:55:11.542567 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:11.542501    6133 scope.go:122] "RemoveContainer" containerID="ac3251b59fa7c86209ea54ce01461512a94cf3ecfb802abd04895d4e0b226da3"
Oct 30 21:55:11.542736 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:11.542718    6133 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ac3251b59fa7c86209ea54ce01461512a94cf3ecfb802abd04895d4e0b226da3"} err="failed to get container status \"ac3251b59fa7c86209ea54ce01461512a94cf3ecfb802abd04895d4e0b226da3\": rpc error: code = NotFound desc = could not find container \"ac3251b59fa7c86209ea54ce01461512a94cf3ecfb802abd04895d4e0b226da3\": container with ID starting with ac3251b59fa7c86209ea54ce01461512a94cf3ecfb802abd04895d4e0b226da3 not found: ID does not exist"
Oct 30 21:55:11.542804 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:11.542737    6133 scope.go:122] "RemoveContainer" containerID="7f5d14c4283a270b49a30f65cd969f9059aa7d43f0f4a78590276fa736dae993"
Oct 30 21:55:11.542965 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:11.542935    6133 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7f5d14c4283a270b49a30f65cd969f9059aa7d43f0f4a78590276fa736dae993"} err="failed to get container status \"7f5d14c4283a270b49a30f65cd969f9059aa7d43f0f4a78590276fa736dae993\": rpc error: code = NotFound desc = could not find container \"7f5d14c4283a270b49a30f65cd969f9059aa7d43f0f4a78590276fa736dae993\": container with ID starting with 7f5d14c4283a270b49a30f65cd969f9059aa7d43f0f4a78590276fa736dae993 not found: ID does not exist"
Oct 30 21:55:11.543026 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:11.542964    6133 scope.go:122] "RemoveContainer" containerID="fc4e2bd2f898fb51f973cbd22a4c415acb3e71bb1ccc34e58f5925bda9c48eda"
Oct 30 21:55:11.543216 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:11.543196    6133 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fc4e2bd2f898fb51f973cbd22a4c415acb3e71bb1ccc34e58f5925bda9c48eda"} err="failed to get container status \"fc4e2bd2f898fb51f973cbd22a4c415acb3e71bb1ccc34e58f5925bda9c48eda\": rpc error: code = NotFound desc = could not find container \"fc4e2bd2f898fb51f973cbd22a4c415acb3e71bb1ccc34e58f5925bda9c48eda\": container with ID starting with fc4e2bd2f898fb51f973cbd22a4c415acb3e71bb1ccc34e58f5925bda9c48eda not found: ID does not exist"
Oct 30 21:55:11.543255 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:11.543216    6133 scope.go:122] "RemoveContainer" containerID="1afbd58205aa89f01dece6ea4aceecdb2c1a95d6f90583bfa5c329026ead89df"
Oct 30 21:55:11.543415 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:11.543396    6133 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1afbd58205aa89f01dece6ea4aceecdb2c1a95d6f90583bfa5c329026ead89df"} err="failed to get container status \"1afbd58205aa89f01dece6ea4aceecdb2c1a95d6f90583bfa5c329026ead89df\": rpc error: code = NotFound desc = could not find container \"1afbd58205aa89f01dece6ea4aceecdb2c1a95d6f90583bfa5c329026ead89df\": container with ID starting with 1afbd58205aa89f01dece6ea4aceecdb2c1a95d6f90583bfa5c329026ead89df not found: ID does not exist"
Oct 30 21:55:11.543461 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:11.543416    6133 scope.go:122] "RemoveContainer" containerID="9915681fb0e03c6fd0b8260c96d61e8c11ce6f9d448cf02897ca8c9db3796828"
Oct 30 21:55:11.543652 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:11.543635    6133 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9915681fb0e03c6fd0b8260c96d61e8c11ce6f9d448cf02897ca8c9db3796828"} err="failed to get container status \"9915681fb0e03c6fd0b8260c96d61e8c11ce6f9d448cf02897ca8c9db3796828\": rpc error: code = NotFound desc = could not find container \"9915681fb0e03c6fd0b8260c96d61e8c11ce6f9d448cf02897ca8c9db3796828\": container with ID starting with 9915681fb0e03c6fd0b8260c96d61e8c11ce6f9d448cf02897ca8c9db3796828 not found: ID does not exist"
Oct 30 21:55:11.543697 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:11.543652    6133 scope.go:122] "RemoveContainer" containerID="4b5654280a0b3c41ca38a375f20091c72976104b1839d8f9f2405f57b1570677"
Oct 30 21:55:11.543857 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:11.543837    6133 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4b5654280a0b3c41ca38a375f20091c72976104b1839d8f9f2405f57b1570677"} err="failed to get container status \"4b5654280a0b3c41ca38a375f20091c72976104b1839d8f9f2405f57b1570677\": rpc error: code = NotFound desc = could not find container \"4b5654280a0b3c41ca38a375f20091c72976104b1839d8f9f2405f57b1570677\": container with ID starting with 4b5654280a0b3c41ca38a375f20091c72976104b1839d8f9f2405f57b1570677 not found: ID does not exist"
Oct 30 21:55:11.543857 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:11.543855    6133 scope.go:122] "RemoveContainer" containerID="d76784ca45eef8daaa0b9110107077e348d4d174313732f2cc153bbbff9d32eb"
Oct 30 21:55:11.544095 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:11.544077    6133 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d76784ca45eef8daaa0b9110107077e348d4d174313732f2cc153bbbff9d32eb"} err="failed to get container status \"d76784ca45eef8daaa0b9110107077e348d4d174313732f2cc153bbbff9d32eb\": rpc error: code = NotFound desc = could not find container \"d76784ca45eef8daaa0b9110107077e348d4d174313732f2cc153bbbff9d32eb\": container with ID starting with d76784ca45eef8daaa0b9110107077e348d4d174313732f2cc153bbbff9d32eb not found: ID does not exist"
Oct 30 21:55:11.544138 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:11.544096    6133 scope.go:122] "RemoveContainer" containerID="ac3251b59fa7c86209ea54ce01461512a94cf3ecfb802abd04895d4e0b226da3"
Oct 30 21:55:11.544317 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:11.544299    6133 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ac3251b59fa7c86209ea54ce01461512a94cf3ecfb802abd04895d4e0b226da3"} err="failed to get container status \"ac3251b59fa7c86209ea54ce01461512a94cf3ecfb802abd04895d4e0b226da3\": rpc error: code = NotFound desc = could not find container \"ac3251b59fa7c86209ea54ce01461512a94cf3ecfb802abd04895d4e0b226da3\": container with ID starting with ac3251b59fa7c86209ea54ce01461512a94cf3ecfb802abd04895d4e0b226da3 not found: ID does not exist"
Oct 30 21:55:11.544367 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:11.544317    6133 scope.go:122] "RemoveContainer" containerID="7f5d14c4283a270b49a30f65cd969f9059aa7d43f0f4a78590276fa736dae993"
Oct 30 21:55:11.544505 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:11.544489    6133 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7f5d14c4283a270b49a30f65cd969f9059aa7d43f0f4a78590276fa736dae993"} err="failed to get container status \"7f5d14c4283a270b49a30f65cd969f9059aa7d43f0f4a78590276fa736dae993\": rpc error: code = NotFound desc = could not find container \"7f5d14c4283a270b49a30f65cd969f9059aa7d43f0f4a78590276fa736dae993\": container with ID starting with 7f5d14c4283a270b49a30f65cd969f9059aa7d43f0f4a78590276fa736dae993 not found: ID does not exist"
Oct 30 21:55:11.544545 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:11.544505    6133 scope.go:122] "RemoveContainer" containerID="fc4e2bd2f898fb51f973cbd22a4c415acb3e71bb1ccc34e58f5925bda9c48eda"
Oct 30 21:55:11.544702 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:11.544682    6133 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fc4e2bd2f898fb51f973cbd22a4c415acb3e71bb1ccc34e58f5925bda9c48eda"} err="failed to get container status \"fc4e2bd2f898fb51f973cbd22a4c415acb3e71bb1ccc34e58f5925bda9c48eda\": rpc error: code = NotFound desc = could not find container \"fc4e2bd2f898fb51f973cbd22a4c415acb3e71bb1ccc34e58f5925bda9c48eda\": container with ID starting with fc4e2bd2f898fb51f973cbd22a4c415acb3e71bb1ccc34e58f5925bda9c48eda not found: ID does not exist"
Oct 30 21:55:11.544749 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:11.544703    6133 scope.go:122] "RemoveContainer" containerID="1afbd58205aa89f01dece6ea4aceecdb2c1a95d6f90583bfa5c329026ead89df"
Oct 30 21:55:11.544887 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:11.544872    6133 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1afbd58205aa89f01dece6ea4aceecdb2c1a95d6f90583bfa5c329026ead89df"} err="failed to get container status \"1afbd58205aa89f01dece6ea4aceecdb2c1a95d6f90583bfa5c329026ead89df\": rpc error: code = NotFound desc = could not find container \"1afbd58205aa89f01dece6ea4aceecdb2c1a95d6f90583bfa5c329026ead89df\": container with ID starting with 1afbd58205aa89f01dece6ea4aceecdb2c1a95d6f90583bfa5c329026ead89df not found: ID does not exist"
Oct 30 21:55:11.544931 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:11.544887    6133 scope.go:122] "RemoveContainer" containerID="9915681fb0e03c6fd0b8260c96d61e8c11ce6f9d448cf02897ca8c9db3796828"
Oct 30 21:55:11.545159 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:11.545144    6133 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9915681fb0e03c6fd0b8260c96d61e8c11ce6f9d448cf02897ca8c9db3796828"} err="failed to get container status \"9915681fb0e03c6fd0b8260c96d61e8c11ce6f9d448cf02897ca8c9db3796828\": rpc error: code = NotFound desc = could not find container \"9915681fb0e03c6fd0b8260c96d61e8c11ce6f9d448cf02897ca8c9db3796828\": container with ID starting with 9915681fb0e03c6fd0b8260c96d61e8c11ce6f9d448cf02897ca8c9db3796828 not found: ID does not exist"
Oct 30 21:55:11.545159 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:11.545157    6133 scope.go:122] "RemoveContainer" containerID="4b5654280a0b3c41ca38a375f20091c72976104b1839d8f9f2405f57b1570677"
Oct 30 21:55:11.545370 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:11.545352    6133 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4b5654280a0b3c41ca38a375f20091c72976104b1839d8f9f2405f57b1570677"} err="failed to get container status \"4b5654280a0b3c41ca38a375f20091c72976104b1839d8f9f2405f57b1570677\": rpc error: code = NotFound desc = could not find container \"4b5654280a0b3c41ca38a375f20091c72976104b1839d8f9f2405f57b1570677\": container with ID starting with 4b5654280a0b3c41ca38a375f20091c72976104b1839d8f9f2405f57b1570677 not found: ID does not exist"
Oct 30 21:55:11.545370 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:11.545365    6133 scope.go:122] "RemoveContainer" containerID="d76784ca45eef8daaa0b9110107077e348d4d174313732f2cc153bbbff9d32eb"
Oct 30 21:55:11.545561 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:11.545544    6133 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d76784ca45eef8daaa0b9110107077e348d4d174313732f2cc153bbbff9d32eb"} err="failed to get container status \"d76784ca45eef8daaa0b9110107077e348d4d174313732f2cc153bbbff9d32eb\": rpc error: code = NotFound desc = could not find container \"d76784ca45eef8daaa0b9110107077e348d4d174313732f2cc153bbbff9d32eb\": container with ID starting with d76784ca45eef8daaa0b9110107077e348d4d174313732f2cc153bbbff9d32eb not found: ID does not exist"
Oct 30 21:55:11.545623 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:11.545561    6133 scope.go:122] "RemoveContainer" containerID="ac3251b59fa7c86209ea54ce01461512a94cf3ecfb802abd04895d4e0b226da3"
Oct 30 21:55:11.545784 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:11.545766    6133 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ac3251b59fa7c86209ea54ce01461512a94cf3ecfb802abd04895d4e0b226da3"} err="failed to get container status \"ac3251b59fa7c86209ea54ce01461512a94cf3ecfb802abd04895d4e0b226da3\": rpc error: code = NotFound desc = could not find container \"ac3251b59fa7c86209ea54ce01461512a94cf3ecfb802abd04895d4e0b226da3\": container with ID starting with ac3251b59fa7c86209ea54ce01461512a94cf3ecfb802abd04895d4e0b226da3 not found: ID does not exist"
Oct 30 21:55:11.545830 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:11.545784    6133 scope.go:122] "RemoveContainer" containerID="7f5d14c4283a270b49a30f65cd969f9059aa7d43f0f4a78590276fa736dae993"
Oct 30 21:55:11.546137 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:11.546119    6133 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7f5d14c4283a270b49a30f65cd969f9059aa7d43f0f4a78590276fa736dae993"} err="failed to get container status \"7f5d14c4283a270b49a30f65cd969f9059aa7d43f0f4a78590276fa736dae993\": rpc error: code = NotFound desc = could not find container \"7f5d14c4283a270b49a30f65cd969f9059aa7d43f0f4a78590276fa736dae993\": container with ID starting with 7f5d14c4283a270b49a30f65cd969f9059aa7d43f0f4a78590276fa736dae993 not found: ID does not exist"
Oct 30 21:55:11.546217 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:11.546137    6133 scope.go:122] "RemoveContainer" containerID="fc4e2bd2f898fb51f973cbd22a4c415acb3e71bb1ccc34e58f5925bda9c48eda"
Oct 30 21:55:11.546372 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:11.546353    6133 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fc4e2bd2f898fb51f973cbd22a4c415acb3e71bb1ccc34e58f5925bda9c48eda"} err="failed to get container status \"fc4e2bd2f898fb51f973cbd22a4c415acb3e71bb1ccc34e58f5925bda9c48eda\": rpc error: code = NotFound desc = could not find container \"fc4e2bd2f898fb51f973cbd22a4c415acb3e71bb1ccc34e58f5925bda9c48eda\": container with ID starting with fc4e2bd2f898fb51f973cbd22a4c415acb3e71bb1ccc34e58f5925bda9c48eda not found: ID does not exist"
Oct 30 21:55:11.546372 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:11.546370    6133 scope.go:122] "RemoveContainer" containerID="1afbd58205aa89f01dece6ea4aceecdb2c1a95d6f90583bfa5c329026ead89df"
Oct 30 21:55:11.546580 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:11.546564    6133 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1afbd58205aa89f01dece6ea4aceecdb2c1a95d6f90583bfa5c329026ead89df"} err="failed to get container status \"1afbd58205aa89f01dece6ea4aceecdb2c1a95d6f90583bfa5c329026ead89df\": rpc error: code = NotFound desc = could not find container \"1afbd58205aa89f01dece6ea4aceecdb2c1a95d6f90583bfa5c329026ead89df\": container with ID starting with 1afbd58205aa89f01dece6ea4aceecdb2c1a95d6f90583bfa5c329026ead89df not found: ID does not exist"
Oct 30 21:55:11.546580 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:11.546579    6133 scope.go:122] "RemoveContainer" containerID="9915681fb0e03c6fd0b8260c96d61e8c11ce6f9d448cf02897ca8c9db3796828"
Oct 30 21:55:11.546788 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:11.546771    6133 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9915681fb0e03c6fd0b8260c96d61e8c11ce6f9d448cf02897ca8c9db3796828"} err="failed to get container status \"9915681fb0e03c6fd0b8260c96d61e8c11ce6f9d448cf02897ca8c9db3796828\": rpc error: code = NotFound desc = could not find container \"9915681fb0e03c6fd0b8260c96d61e8c11ce6f9d448cf02897ca8c9db3796828\": container with ID starting with 9915681fb0e03c6fd0b8260c96d61e8c11ce6f9d448cf02897ca8c9db3796828 not found: ID does not exist"
Oct 30 21:55:11.546831 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:11.546788    6133 scope.go:122] "RemoveContainer" containerID="4b5654280a0b3c41ca38a375f20091c72976104b1839d8f9f2405f57b1570677"
Oct 30 21:55:11.547015 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:11.546996    6133 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4b5654280a0b3c41ca38a375f20091c72976104b1839d8f9f2405f57b1570677"} err="failed to get container status \"4b5654280a0b3c41ca38a375f20091c72976104b1839d8f9f2405f57b1570677\": rpc error: code = NotFound desc = could not find container \"4b5654280a0b3c41ca38a375f20091c72976104b1839d8f9f2405f57b1570677\": container with ID starting with 4b5654280a0b3c41ca38a375f20091c72976104b1839d8f9f2405f57b1570677 not found: ID does not exist"
Oct 30 21:55:11.547061 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:11.547016    6133 scope.go:122] "RemoveContainer" containerID="d76784ca45eef8daaa0b9110107077e348d4d174313732f2cc153bbbff9d32eb"
Oct 30 21:55:11.547232 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:11.547214    6133 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d76784ca45eef8daaa0b9110107077e348d4d174313732f2cc153bbbff9d32eb"} err="failed to get container status \"d76784ca45eef8daaa0b9110107077e348d4d174313732f2cc153bbbff9d32eb\": rpc error: code = NotFound desc = could not find container \"d76784ca45eef8daaa0b9110107077e348d4d174313732f2cc153bbbff9d32eb\": container with ID starting with d76784ca45eef8daaa0b9110107077e348d4d174313732f2cc153bbbff9d32eb not found: ID does not exist"
Oct 30 21:55:11.547272 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:11.547234    6133 scope.go:122] "RemoveContainer" containerID="ac3251b59fa7c86209ea54ce01461512a94cf3ecfb802abd04895d4e0b226da3"
Oct 30 21:55:11.547441 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:11.547419    6133 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ac3251b59fa7c86209ea54ce01461512a94cf3ecfb802abd04895d4e0b226da3"} err="failed to get container status \"ac3251b59fa7c86209ea54ce01461512a94cf3ecfb802abd04895d4e0b226da3\": rpc error: code = NotFound desc = could not find container \"ac3251b59fa7c86209ea54ce01461512a94cf3ecfb802abd04895d4e0b226da3\": container with ID starting with ac3251b59fa7c86209ea54ce01461512a94cf3ecfb802abd04895d4e0b226da3 not found: ID does not exist"
Oct 30 21:55:11.547490 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:11.547440    6133 scope.go:122] "RemoveContainer" containerID="7f5d14c4283a270b49a30f65cd969f9059aa7d43f0f4a78590276fa736dae993"
Oct 30 21:55:11.547655 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:11.547635    6133 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7f5d14c4283a270b49a30f65cd969f9059aa7d43f0f4a78590276fa736dae993"} err="failed to get container status \"7f5d14c4283a270b49a30f65cd969f9059aa7d43f0f4a78590276fa736dae993\": rpc error: code = NotFound desc = could not find container \"7f5d14c4283a270b49a30f65cd969f9059aa7d43f0f4a78590276fa736dae993\": container with ID starting with 7f5d14c4283a270b49a30f65cd969f9059aa7d43f0f4a78590276fa736dae993 not found: ID does not exist"
Oct 30 21:55:11.547718 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:11.547655    6133 scope.go:122] "RemoveContainer" containerID="fc4e2bd2f898fb51f973cbd22a4c415acb3e71bb1ccc34e58f5925bda9c48eda"
Oct 30 21:55:11.547901 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:11.547881    6133 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fc4e2bd2f898fb51f973cbd22a4c415acb3e71bb1ccc34e58f5925bda9c48eda"} err="failed to get container status \"fc4e2bd2f898fb51f973cbd22a4c415acb3e71bb1ccc34e58f5925bda9c48eda\": rpc error: code = NotFound desc = could not find container \"fc4e2bd2f898fb51f973cbd22a4c415acb3e71bb1ccc34e58f5925bda9c48eda\": container with ID starting with fc4e2bd2f898fb51f973cbd22a4c415acb3e71bb1ccc34e58f5925bda9c48eda not found: ID does not exist"
Oct 30 21:55:11.547967 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:11.547902    6133 scope.go:122] "RemoveContainer" containerID="1afbd58205aa89f01dece6ea4aceecdb2c1a95d6f90583bfa5c329026ead89df"
Oct 30 21:55:11.548172 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:11.548154    6133 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1afbd58205aa89f01dece6ea4aceecdb2c1a95d6f90583bfa5c329026ead89df"} err="failed to get container status \"1afbd58205aa89f01dece6ea4aceecdb2c1a95d6f90583bfa5c329026ead89df\": rpc error: code = NotFound desc = could not find container \"1afbd58205aa89f01dece6ea4aceecdb2c1a95d6f90583bfa5c329026ead89df\": container with ID starting with 1afbd58205aa89f01dece6ea4aceecdb2c1a95d6f90583bfa5c329026ead89df not found: ID does not exist"
Oct 30 21:55:11.628576 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:11.628494    6133 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/bd93518f-2f96-4483-9e92-513109dd37cb-config-out\") on node \"ip-10-0-143-246.ec2.internal\" DevicePath \"\""
Oct 30 21:55:11.628576 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:11.628521    6133 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/bd93518f-2f96-4483-9e92-513109dd37cb-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-143-246.ec2.internal\" DevicePath \"\""
Oct 30 21:55:11.628576 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:11.628532    6133 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/bd93518f-2f96-4483-9e92-513109dd37cb-web-config\") on node \"ip-10-0-143-246.ec2.internal\" DevicePath \"\""
Oct 30 21:55:11.628576 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:11.628545    6133 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/bd93518f-2f96-4483-9e92-513109dd37cb-secret-kube-rbac-proxy\") on node \"ip-10-0-143-246.ec2.internal\" DevicePath \"\""
Oct 30 21:55:11.628576 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:11.628554    6133 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/bd93518f-2f96-4483-9e92-513109dd37cb-secret-prometheus-k8s-tls\") on node \"ip-10-0-143-246.ec2.internal\" DevicePath \"\""
Oct 30 21:55:11.628576 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:11.628565    6133 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/bd93518f-2f96-4483-9e92-513109dd37cb-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-143-246.ec2.internal\" DevicePath \"\""
Oct 30 21:55:11.628576 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:11.628574    6133 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/bd93518f-2f96-4483-9e92-513109dd37cb-secret-metrics-client-certs\") on node \"ip-10-0-143-246.ec2.internal\" DevicePath \"\""
Oct 30 21:55:11.628576 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:11.628583    6133 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/bd93518f-2f96-4483-9e92-513109dd37cb-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-143-246.ec2.internal\" DevicePath \"\""
Oct 30 21:55:11.628928 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:11.628592    6133 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/bd93518f-2f96-4483-9e92-513109dd37cb-thanos-prometheus-http-client-file\") on node \"ip-10-0-143-246.ec2.internal\" DevicePath \"\""
Oct 30 21:55:11.628928 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:11.628602    6133 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/bd93518f-2f96-4483-9e92-513109dd37cb-config\") on node \"ip-10-0-143-246.ec2.internal\" DevicePath \"\""
Oct 30 21:55:11.628928 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:11.628615    6133 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/bd93518f-2f96-4483-9e92-513109dd37cb-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-143-246.ec2.internal\" DevicePath \"\""
Oct 30 21:55:11.628928 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:11.628626    6133 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/bd93518f-2f96-4483-9e92-513109dd37cb-tls-assets\") on node \"ip-10-0-143-246.ec2.internal\" DevicePath \"\""
Oct 30 21:55:11.628928 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:11.628634    6133 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qrt69\" (UniqueName: \"kubernetes.io/projected/bd93518f-2f96-4483-9e92-513109dd37cb-kube-api-access-qrt69\") on node \"ip-10-0-143-246.ec2.internal\" DevicePath \"\""
Oct 30 21:55:11.842142 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:11.842101    6133 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Oct 30 21:55:11.854772 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:11.854739    6133 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Oct 30 21:55:11.893581 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:11.893505    6133 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Oct 30 21:55:11.893799 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:11.893786    6133 cpu_manager.go:414] "containerMap: removing container" podUID="bd93518f-2f96-4483-9e92-513109dd37cb" containerName="init-config-reloader"
Oct 30 21:55:11.893844 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:11.893800    6133 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="bd93518f-2f96-4483-9e92-513109dd37cb" containerName="init-config-reloader"
Oct 30 21:55:11.893844 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:11.893810    6133 cpu_manager.go:414] "containerMap: removing container" podUID="bd93518f-2f96-4483-9e92-513109dd37cb" containerName="kube-rbac-proxy"
Oct 30 21:55:11.893844 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:11.893815    6133 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="bd93518f-2f96-4483-9e92-513109dd37cb" containerName="kube-rbac-proxy"
Oct 30 21:55:11.893844 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:11.893824    6133 cpu_manager.go:414] "containerMap: removing container" podUID="bd93518f-2f96-4483-9e92-513109dd37cb" containerName="kube-rbac-proxy-web"
Oct 30 21:55:11.893844 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:11.893830    6133 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="bd93518f-2f96-4483-9e92-513109dd37cb" containerName="kube-rbac-proxy-web"
Oct 30 21:55:11.893844 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:11.893841    6133 cpu_manager.go:414] "containerMap: removing container" podUID="bd93518f-2f96-4483-9e92-513109dd37cb" containerName="thanos-sidecar"
Oct 30 21:55:11.893844 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:11.893845    6133 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="bd93518f-2f96-4483-9e92-513109dd37cb" containerName="thanos-sidecar"
Oct 30 21:55:11.894066 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:11.893853    6133 cpu_manager.go:414] "containerMap: removing container" podUID="bd93518f-2f96-4483-9e92-513109dd37cb" containerName="prometheus"
Oct 30 21:55:11.894066 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:11.893858    6133 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="bd93518f-2f96-4483-9e92-513109dd37cb" containerName="prometheus"
Oct 30 21:55:11.894066 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:11.893875    6133 cpu_manager.go:414] "containerMap: removing container" podUID="bd93518f-2f96-4483-9e92-513109dd37cb" containerName="config-reloader"
Oct 30 21:55:11.894066 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:11.893879    6133 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="bd93518f-2f96-4483-9e92-513109dd37cb" containerName="config-reloader"
Oct 30 21:55:11.894066 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:11.893887    6133 cpu_manager.go:414] "containerMap: removing container" podUID="bd93518f-2f96-4483-9e92-513109dd37cb" containerName="kube-rbac-proxy-thanos"
Oct 30 21:55:11.894066 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:11.893892    6133 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="bd93518f-2f96-4483-9e92-513109dd37cb" containerName="kube-rbac-proxy-thanos"
Oct 30 21:55:11.894066 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:11.893930    6133 memory_manager.go:365] "RemoveStaleState removing state" podUID="bd93518f-2f96-4483-9e92-513109dd37cb" containerName="config-reloader"
Oct 30 21:55:11.894066 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:11.893937    6133 memory_manager.go:365] "RemoveStaleState removing state" podUID="bd93518f-2f96-4483-9e92-513109dd37cb" containerName="kube-rbac-proxy-thanos"
Oct 30 21:55:11.894066 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:11.893944    6133 memory_manager.go:365] "RemoveStaleState removing state" podUID="bd93518f-2f96-4483-9e92-513109dd37cb" containerName="init-config-reloader"
Oct 30 21:55:11.894066 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:11.893950    6133 memory_manager.go:365] "RemoveStaleState removing state" podUID="bd93518f-2f96-4483-9e92-513109dd37cb" containerName="kube-rbac-proxy"
Oct 30 21:55:11.894066 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:11.893955    6133 memory_manager.go:365] "RemoveStaleState removing state" podUID="bd93518f-2f96-4483-9e92-513109dd37cb" containerName="kube-rbac-proxy-web"
Oct 30 21:55:11.894066 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:11.893961    6133 memory_manager.go:365] "RemoveStaleState removing state" podUID="bd93518f-2f96-4483-9e92-513109dd37cb" containerName="thanos-sidecar"
Oct 30 21:55:11.894066 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:11.893965    6133 memory_manager.go:365] "RemoveStaleState removing state" podUID="bd93518f-2f96-4483-9e92-513109dd37cb" containerName="prometheus"
Oct 30 21:55:11.897298 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:11.897282    6133 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:55:11.914906 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:11.914879    6133 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Oct 30 21:55:11.914906 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:11.914879    6133 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Oct 30 21:55:11.915218 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:11.915202    6133 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Oct 30 21:55:11.915273 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:11.915235    6133 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Oct 30 21:55:11.915273 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:11.915254    6133 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Oct 30 21:55:11.915369 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:11.915267    6133 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Oct 30 21:55:11.915369 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:11.915306    6133 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Oct 30 21:55:11.915369 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:11.915321    6133 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Oct 30 21:55:11.915369 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:11.915333    6133 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Oct 30 21:55:11.915369 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:11.915357    6133 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-9k9apv9hqg1mm\""
Oct 30 21:55:11.915520 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:11.915404    6133 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-xwsb9\""
Oct 30 21:55:11.917884 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:11.917868    6133 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Oct 30 21:55:11.918016 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:11.917982    6133 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Oct 30 21:55:11.918217 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:11.918200    6133 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Oct 30 21:55:11.918217 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:11.918213    6133 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Oct 30 21:55:11.931923 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:11.931891    6133 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Oct 30 21:55:11.932106 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:11.932011    6133 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Oct 30 21:55:11.932516 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:11.932496    6133 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Oct 30 21:55:12.032450 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:12.032416    6133 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f77c55ab-9c27-4472-ab3b-83c11b803db0-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"f77c55ab-9c27-4472-ab3b-83c11b803db0\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:55:12.032450 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:12.032452    6133 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/f77c55ab-9c27-4472-ab3b-83c11b803db0-config\") pod \"prometheus-k8s-0\" (UID: \"f77c55ab-9c27-4472-ab3b-83c11b803db0\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:55:12.032714 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:12.032474    6133 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f77c55ab-9c27-4472-ab3b-83c11b803db0-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f77c55ab-9c27-4472-ab3b-83c11b803db0\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:55:12.032714 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:12.032536    6133 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/f77c55ab-9c27-4472-ab3b-83c11b803db0-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"f77c55ab-9c27-4472-ab3b-83c11b803db0\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:55:12.032714 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:12.032556    6133 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f77c55ab-9c27-4472-ab3b-83c11b803db0-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f77c55ab-9c27-4472-ab3b-83c11b803db0\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:55:12.032714 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:12.032691    6133 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f77c55ab-9c27-4472-ab3b-83c11b803db0-web-config\") pod \"prometheus-k8s-0\" (UID: \"f77c55ab-9c27-4472-ab3b-83c11b803db0\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:55:12.032876 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:12.032734    6133 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/f77c55ab-9c27-4472-ab3b-83c11b803db0-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"f77c55ab-9c27-4472-ab3b-83c11b803db0\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:55:12.032876 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:12.032783    6133 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6pblq\" (UniqueName: \"kubernetes.io/projected/f77c55ab-9c27-4472-ab3b-83c11b803db0-kube-api-access-6pblq\") pod \"prometheus-k8s-0\" (UID: \"f77c55ab-9c27-4472-ab3b-83c11b803db0\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:55:12.032876 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:12.032855    6133 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/f77c55ab-9c27-4472-ab3b-83c11b803db0-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"f77c55ab-9c27-4472-ab3b-83c11b803db0\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:55:12.032983 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:12.032879    6133 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f77c55ab-9c27-4472-ab3b-83c11b803db0-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"f77c55ab-9c27-4472-ab3b-83c11b803db0\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:55:12.032983 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:12.032901    6133 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f77c55ab-9c27-4472-ab3b-83c11b803db0-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"f77c55ab-9c27-4472-ab3b-83c11b803db0\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:55:12.032983 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:12.032943    6133 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/f77c55ab-9c27-4472-ab3b-83c11b803db0-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"f77c55ab-9c27-4472-ab3b-83c11b803db0\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:55:12.032983 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:12.032964    6133 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/f77c55ab-9c27-4472-ab3b-83c11b803db0-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"f77c55ab-9c27-4472-ab3b-83c11b803db0\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:55:12.032983 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:12.032981    6133 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/f77c55ab-9c27-4472-ab3b-83c11b803db0-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"f77c55ab-9c27-4472-ab3b-83c11b803db0\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:55:12.033198 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:12.033046    6133 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/f77c55ab-9c27-4472-ab3b-83c11b803db0-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"f77c55ab-9c27-4472-ab3b-83c11b803db0\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:55:12.033198 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:12.033117    6133 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f77c55ab-9c27-4472-ab3b-83c11b803db0-config-out\") pod \"prometheus-k8s-0\" (UID: \"f77c55ab-9c27-4472-ab3b-83c11b803db0\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:55:12.033198 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:12.033135    6133 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/f77c55ab-9c27-4472-ab3b-83c11b803db0-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"f77c55ab-9c27-4472-ab3b-83c11b803db0\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:55:12.033198 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:12.033175    6133 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f77c55ab-9c27-4472-ab3b-83c11b803db0-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"f77c55ab-9c27-4472-ab3b-83c11b803db0\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:55:12.033198 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:12.033193    6133 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f77c55ab-9c27-4472-ab3b-83c11b803db0-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f77c55ab-9c27-4472-ab3b-83c11b803db0\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:55:12.033366 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:12.033207    6133 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/f77c55ab-9c27-4472-ab3b-83c11b803db0-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"f77c55ab-9c27-4472-ab3b-83c11b803db0\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:55:12.133795 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:12.133757    6133 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6pblq\" (UniqueName: \"kubernetes.io/projected/f77c55ab-9c27-4472-ab3b-83c11b803db0-kube-api-access-6pblq\") pod \"prometheus-k8s-0\" (UID: \"f77c55ab-9c27-4472-ab3b-83c11b803db0\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:55:12.134270 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:12.133804    6133 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/f77c55ab-9c27-4472-ab3b-83c11b803db0-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"f77c55ab-9c27-4472-ab3b-83c11b803db0\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:55:12.134270 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:12.133824    6133 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f77c55ab-9c27-4472-ab3b-83c11b803db0-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"f77c55ab-9c27-4472-ab3b-83c11b803db0\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:55:12.134270 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:12.133842    6133 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f77c55ab-9c27-4472-ab3b-83c11b803db0-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"f77c55ab-9c27-4472-ab3b-83c11b803db0\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:55:12.134270 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:12.133966    6133 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/f77c55ab-9c27-4472-ab3b-83c11b803db0-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"f77c55ab-9c27-4472-ab3b-83c11b803db0\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:55:12.134270 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:12.134035    6133 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/f77c55ab-9c27-4472-ab3b-83c11b803db0-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"f77c55ab-9c27-4472-ab3b-83c11b803db0\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:55:12.134270 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:12.134069    6133 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/f77c55ab-9c27-4472-ab3b-83c11b803db0-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"f77c55ab-9c27-4472-ab3b-83c11b803db0\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:55:12.134270 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:12.134098    6133 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/f77c55ab-9c27-4472-ab3b-83c11b803db0-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"f77c55ab-9c27-4472-ab3b-83c11b803db0\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:55:12.134270 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:12.134130    6133 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f77c55ab-9c27-4472-ab3b-83c11b803db0-config-out\") pod \"prometheus-k8s-0\" (UID: \"f77c55ab-9c27-4472-ab3b-83c11b803db0\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:55:12.134270 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:12.134250    6133 swap_util.go:78] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Oct 30 21:55:12.134270 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:12.134261    6133 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/f77c55ab-9c27-4472-ab3b-83c11b803db0-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"f77c55ab-9c27-4472-ab3b-83c11b803db0\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:55:12.134771 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:12.134295    6133 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f77c55ab-9c27-4472-ab3b-83c11b803db0-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"f77c55ab-9c27-4472-ab3b-83c11b803db0\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:55:12.134771 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:12.134326    6133 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f77c55ab-9c27-4472-ab3b-83c11b803db0-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f77c55ab-9c27-4472-ab3b-83c11b803db0\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:55:12.134771 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:12.134353    6133 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/f77c55ab-9c27-4472-ab3b-83c11b803db0-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"f77c55ab-9c27-4472-ab3b-83c11b803db0\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:55:12.134771 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:12.134409    6133 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f77c55ab-9c27-4472-ab3b-83c11b803db0-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"f77c55ab-9c27-4472-ab3b-83c11b803db0\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:55:12.134771 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:12.134437    6133 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/f77c55ab-9c27-4472-ab3b-83c11b803db0-config\") pod \"prometheus-k8s-0\" (UID: \"f77c55ab-9c27-4472-ab3b-83c11b803db0\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:55:12.134771 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:12.134465    6133 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f77c55ab-9c27-4472-ab3b-83c11b803db0-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f77c55ab-9c27-4472-ab3b-83c11b803db0\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:55:12.134771 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:12.134500    6133 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/f77c55ab-9c27-4472-ab3b-83c11b803db0-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"f77c55ab-9c27-4472-ab3b-83c11b803db0\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:55:12.134771 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:12.134537    6133 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f77c55ab-9c27-4472-ab3b-83c11b803db0-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f77c55ab-9c27-4472-ab3b-83c11b803db0\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:55:12.134771 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:12.134565    6133 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f77c55ab-9c27-4472-ab3b-83c11b803db0-web-config\") pod \"prometheus-k8s-0\" (UID: \"f77c55ab-9c27-4472-ab3b-83c11b803db0\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:55:12.134771 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:12.134600    6133 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/f77c55ab-9c27-4472-ab3b-83c11b803db0-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"f77c55ab-9c27-4472-ab3b-83c11b803db0\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:55:12.134771 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:12.134651    6133 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f77c55ab-9c27-4472-ab3b-83c11b803db0-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"f77c55ab-9c27-4472-ab3b-83c11b803db0\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:55:12.135365 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:12.134896    6133 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/f77c55ab-9c27-4472-ab3b-83c11b803db0-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"f77c55ab-9c27-4472-ab3b-83c11b803db0\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:55:12.137296 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:12.136902    6133 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f77c55ab-9c27-4472-ab3b-83c11b803db0-config-out\") pod \"prometheus-k8s-0\" (UID: \"f77c55ab-9c27-4472-ab3b-83c11b803db0\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:55:12.137296 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:12.137256    6133 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/f77c55ab-9c27-4472-ab3b-83c11b803db0-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"f77c55ab-9c27-4472-ab3b-83c11b803db0\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:55:12.137296 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:12.137263    6133 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/f77c55ab-9c27-4472-ab3b-83c11b803db0-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"f77c55ab-9c27-4472-ab3b-83c11b803db0\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:55:12.137538 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:12.137307    6133 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/f77c55ab-9c27-4472-ab3b-83c11b803db0-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"f77c55ab-9c27-4472-ab3b-83c11b803db0\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:55:12.137538 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:12.137459    6133 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f77c55ab-9c27-4472-ab3b-83c11b803db0-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"f77c55ab-9c27-4472-ab3b-83c11b803db0\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:55:12.137660 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:12.137621    6133 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/f77c55ab-9c27-4472-ab3b-83c11b803db0-config\") pod \"prometheus-k8s-0\" (UID: \"f77c55ab-9c27-4472-ab3b-83c11b803db0\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:55:12.137875 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:12.137849    6133 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/f77c55ab-9c27-4472-ab3b-83c11b803db0-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"f77c55ab-9c27-4472-ab3b-83c11b803db0\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:55:12.137952 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:12.137908    6133 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f77c55ab-9c27-4472-ab3b-83c11b803db0-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f77c55ab-9c27-4472-ab3b-83c11b803db0\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:55:12.138741 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:12.138185    6133 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f77c55ab-9c27-4472-ab3b-83c11b803db0-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f77c55ab-9c27-4472-ab3b-83c11b803db0\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:55:12.138741 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:12.138360    6133 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/f77c55ab-9c27-4472-ab3b-83c11b803db0-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"f77c55ab-9c27-4472-ab3b-83c11b803db0\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:55:12.138741 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:12.138688    6133 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/f77c55ab-9c27-4472-ab3b-83c11b803db0-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"f77c55ab-9c27-4472-ab3b-83c11b803db0\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:55:12.138975 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:12.138808    6133 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f77c55ab-9c27-4472-ab3b-83c11b803db0-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f77c55ab-9c27-4472-ab3b-83c11b803db0\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:55:12.139113 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:12.139095    6133 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f77c55ab-9c27-4472-ab3b-83c11b803db0-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"f77c55ab-9c27-4472-ab3b-83c11b803db0\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:55:12.139331 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:12.139309    6133 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/f77c55ab-9c27-4472-ab3b-83c11b803db0-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"f77c55ab-9c27-4472-ab3b-83c11b803db0\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:55:12.139477 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:12.139457    6133 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f77c55ab-9c27-4472-ab3b-83c11b803db0-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"f77c55ab-9c27-4472-ab3b-83c11b803db0\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:55:12.140104 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:12.140085    6133 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f77c55ab-9c27-4472-ab3b-83c11b803db0-web-config\") pod \"prometheus-k8s-0\" (UID: \"f77c55ab-9c27-4472-ab3b-83c11b803db0\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:55:12.140279 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:12.140263    6133 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/f77c55ab-9c27-4472-ab3b-83c11b803db0-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"f77c55ab-9c27-4472-ab3b-83c11b803db0\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:55:12.155821 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:12.155798    6133 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6pblq\" (UniqueName: \"kubernetes.io/projected/f77c55ab-9c27-4472-ab3b-83c11b803db0-kube-api-access-6pblq\") pod \"prometheus-k8s-0\" (UID: \"f77c55ab-9c27-4472-ab3b-83c11b803db0\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:55:12.214207 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:12.214167    6133 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:55:12.535565 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:12.535537    6133 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Oct 30 21:55:12.536825 ip-10-0-143-246 kubenswrapper[6133]: W1030 21:55:12.536799    6133 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf77c55ab_9c27_4472_ab3b_83c11b803db0.slice/crio-7c0e38716931e164d57d98f35f08d04d902794eb3492a9a6df82a97a90202d97 WatchSource:0}: Error finding container 7c0e38716931e164d57d98f35f08d04d902794eb3492a9a6df82a97a90202d97: Status 404 returned error can't find the container with id 7c0e38716931e164d57d98f35f08d04d902794eb3492a9a6df82a97a90202d97
Oct 30 21:55:13.354847 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:13.354810    6133 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="bd93518f-2f96-4483-9e92-513109dd37cb" path="/var/lib/kubelet/pods/bd93518f-2f96-4483-9e92-513109dd37cb/volumes"
Oct 30 21:55:13.491140 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:13.491103    6133 generic.go:356] "Generic (PLEG): container finished" podID="f77c55ab-9c27-4472-ab3b-83c11b803db0" containerID="c2f6db50c46eb923a57741c39c3b65f62a34793ca0df161742516a5cee067a81" exitCode=0
Oct 30 21:55:13.491304 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:13.491189    6133 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f77c55ab-9c27-4472-ab3b-83c11b803db0","Type":"ContainerDied","Data":"c2f6db50c46eb923a57741c39c3b65f62a34793ca0df161742516a5cee067a81"}
Oct 30 21:55:13.491304 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:13.491262    6133 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f77c55ab-9c27-4472-ab3b-83c11b803db0","Type":"ContainerStarted","Data":"7c0e38716931e164d57d98f35f08d04d902794eb3492a9a6df82a97a90202d97"}
Oct 30 21:55:14.498421 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:14.498381    6133 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f77c55ab-9c27-4472-ab3b-83c11b803db0","Type":"ContainerStarted","Data":"c0c5541fe3d5ff3f7bf537dcc96f8d93796f9d1fca5f306347336e8bf66b9458"}
Oct 30 21:55:14.498917 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:14.498471    6133 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f77c55ab-9c27-4472-ab3b-83c11b803db0","Type":"ContainerStarted","Data":"90707b7324d7587d3185d9ca1254dcbc6576c4a6584f49384f6b4c7a4b513cf6"}
Oct 30 21:55:14.498917 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:14.498496    6133 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f77c55ab-9c27-4472-ab3b-83c11b803db0","Type":"ContainerStarted","Data":"c7a3992dfe573a75f8918120436f6518cead1ba6d633d85be7795f0dede0cb53"}
Oct 30 21:55:14.498917 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:14.498521    6133 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f77c55ab-9c27-4472-ab3b-83c11b803db0","Type":"ContainerStarted","Data":"839792d836a03d57bb7c523462f6ecd634f352d2c4954671a0bf00ea0fdd1679"}
Oct 30 21:55:14.498917 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:14.498544    6133 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f77c55ab-9c27-4472-ab3b-83c11b803db0","Type":"ContainerStarted","Data":"7fb0bf99cd708d0a3105e5f952a9bb513703158eb71450ca31f3356bc33762e1"}
Oct 30 21:55:14.498917 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:14.498567    6133 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f77c55ab-9c27-4472-ab3b-83c11b803db0","Type":"ContainerStarted","Data":"a1786aa8124acd2812f28afac350dc4a299328c0b01c4991fca11585b51c1868"}
Oct 30 21:55:14.579392 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:14.579338    6133 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=3.564033127 podStartE2EDuration="3.579323461s" totalImagesPullingTime="0s" totalInitContainerRuntime="15.290334ms" isStatefulPod=false podCreationTimestamp="2026-10-30 21:55:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:55:14.566505562 +0000 UTC m=+21.327261451" watchObservedRunningTime="2026-10-30 21:55:14.579323461 +0000 UTC m=+21.340079346"
Oct 30 21:55:17.214615 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:17.214573    6133 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:55:22.888828 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:22.888798    6133 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-thksb"
Oct 30 21:55:23.614276 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:23.614246    6133 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-fvbz8"
Oct 30 21:55:39.527297 ip-10-0-143-246 kubenswrapper[6133]: E1030 21:55:39.527258    6133 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6jppj" containerName="extractor"
Oct 30 21:55:40.895626 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:40.895579    6133 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:55:40.911269 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:40.911241    6133 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:55:41.609001 ip-10-0-143-246 kubenswrapper[6133]: I1030 21:55:41.608961    6133 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:56:09.527268 ip-10-0-143-246 kubenswrapper[6133]: E1030 21:56:09.527224    6133 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6jppj" containerName="extractor"
Oct 30 21:56:39.527710 ip-10-0-143-246 kubenswrapper[6133]: E1030 21:56:39.527620    6133 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6jppj" containerName="extractor"
Oct 30 21:57:09.527227 ip-10-0-143-246 kubenswrapper[6133]: E1030 21:57:09.527184    6133 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6jppj" containerName="extractor"
Oct 30 21:57:39.526864 ip-10-0-143-246 kubenswrapper[6133]: E1030 21:57:39.526825    6133 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6jppj" containerName="extractor"
Oct 30 21:58:09.526692 ip-10-0-143-246 kubenswrapper[6133]: E1030 21:58:09.526600    6133 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6jppj" containerName="extractor"
Oct 30 21:58:39.527308 ip-10-0-143-246 kubenswrapper[6133]: E1030 21:58:39.527268    6133 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6jppj" containerName="extractor"
Oct 30 21:59:09.526922 ip-10-0-143-246 kubenswrapper[6133]: E1030 21:59:09.526879    6133 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6jppj" containerName="extractor"
Oct 30 21:59:39.528647 ip-10-0-143-246 kubenswrapper[6133]: E1030 21:59:39.528561    6133 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6jppj" containerName="extractor"
Oct 30 22:00:09.527820 ip-10-0-143-246 kubenswrapper[6133]: E1030 22:00:09.527777    6133 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6jppj" containerName="extractor"
Oct 30 22:00:39.529808 ip-10-0-143-246 kubenswrapper[6133]: E1030 22:00:39.529763    6133 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6jppj" containerName="extractor"
Oct 30 22:01:09.527663 ip-10-0-143-246 kubenswrapper[6133]: E1030 22:01:09.527576    6133 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6jppj" containerName="extractor"
Oct 30 22:01:39.527221 ip-10-0-143-246 kubenswrapper[6133]: E1030 22:01:39.527180    6133 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6jppj" containerName="extractor"
Oct 30 22:02:09.527120 ip-10-0-143-246 kubenswrapper[6133]: E1030 22:02:09.527082    6133 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6jppj" containerName="extractor"
Oct 30 22:02:39.526693 ip-10-0-143-246 kubenswrapper[6133]: E1030 22:02:39.526611    6133 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6jppj" containerName="extractor"
Oct 30 22:03:09.526790 ip-10-0-143-246 kubenswrapper[6133]: E1030 22:03:09.526750    6133 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6jppj" containerName="extractor"
Oct 30 22:03:39.526910 ip-10-0-143-246 kubenswrapper[6133]: E1030 22:03:39.526859    6133 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6jppj" containerName="extractor"
Oct 30 22:04:09.528171 ip-10-0-143-246 kubenswrapper[6133]: E1030 22:04:09.528058    6133 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6jppj" containerName="extractor"
Oct 30 22:04:38.559835 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:04:38.559801    6133 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kserve/odh-model-controller-7f9f8cdc65-5dzzj"]
Oct 30 22:04:38.563296 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:04:38.563276    6133 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/odh-model-controller-7f9f8cdc65-5dzzj"
Oct 30 22:04:38.580100 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:04:38.580059    6133 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"openshift-service-ca.crt\""
Oct 30 22:04:38.587258 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:04:38.587220    6133 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"kube-root-ca.crt\""
Oct 30 22:04:38.587911 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:04:38.587886    6133 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kserve/odh-model-controller-7f9f8cdc65-5dzzj"]
Oct 30 22:04:38.590670 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:04:38.590648    6133 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"odh-model-controller-webhook-cert\""
Oct 30 22:04:38.590779 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:04:38.590723    6133 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"odh-model-controller-dockercfg-rdpf7\""
Oct 30 22:04:38.590779 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:04:38.590733    6133 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"odh-model-controller-metrics-tls\""
Oct 30 22:04:38.621878 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:04:38.621836    6133 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ff6f5d92-c6a9-41d4-97f1-c646bd6291fe-metrics-certs\") pod \"odh-model-controller-7f9f8cdc65-5dzzj\" (UID: \"ff6f5d92-c6a9-41d4-97f1-c646bd6291fe\") " pod="kserve/odh-model-controller-7f9f8cdc65-5dzzj"
Oct 30 22:04:38.622093 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:04:38.621912    6133 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ff6f5d92-c6a9-41d4-97f1-c646bd6291fe-cert\") pod \"odh-model-controller-7f9f8cdc65-5dzzj\" (UID: \"ff6f5d92-c6a9-41d4-97f1-c646bd6291fe\") " pod="kserve/odh-model-controller-7f9f8cdc65-5dzzj"
Oct 30 22:04:38.622093 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:04:38.622024    6133 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h5gqk\" (UniqueName: \"kubernetes.io/projected/ff6f5d92-c6a9-41d4-97f1-c646bd6291fe-kube-api-access-h5gqk\") pod \"odh-model-controller-7f9f8cdc65-5dzzj\" (UID: \"ff6f5d92-c6a9-41d4-97f1-c646bd6291fe\") " pod="kserve/odh-model-controller-7f9f8cdc65-5dzzj"
Oct 30 22:04:38.723203 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:04:38.723170    6133 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ff6f5d92-c6a9-41d4-97f1-c646bd6291fe-metrics-certs\") pod \"odh-model-controller-7f9f8cdc65-5dzzj\" (UID: \"ff6f5d92-c6a9-41d4-97f1-c646bd6291fe\") " pod="kserve/odh-model-controller-7f9f8cdc65-5dzzj"
Oct 30 22:04:38.723391 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:04:38.723220    6133 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ff6f5d92-c6a9-41d4-97f1-c646bd6291fe-cert\") pod \"odh-model-controller-7f9f8cdc65-5dzzj\" (UID: \"ff6f5d92-c6a9-41d4-97f1-c646bd6291fe\") " pod="kserve/odh-model-controller-7f9f8cdc65-5dzzj"
Oct 30 22:04:38.723391 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:04:38.723264    6133 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h5gqk\" (UniqueName: \"kubernetes.io/projected/ff6f5d92-c6a9-41d4-97f1-c646bd6291fe-kube-api-access-h5gqk\") pod \"odh-model-controller-7f9f8cdc65-5dzzj\" (UID: \"ff6f5d92-c6a9-41d4-97f1-c646bd6291fe\") " pod="kserve/odh-model-controller-7f9f8cdc65-5dzzj"
Oct 30 22:04:38.725730 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:04:38.725701    6133 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ff6f5d92-c6a9-41d4-97f1-c646bd6291fe-cert\") pod \"odh-model-controller-7f9f8cdc65-5dzzj\" (UID: \"ff6f5d92-c6a9-41d4-97f1-c646bd6291fe\") " pod="kserve/odh-model-controller-7f9f8cdc65-5dzzj"
Oct 30 22:04:38.725843 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:04:38.725752    6133 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ff6f5d92-c6a9-41d4-97f1-c646bd6291fe-metrics-certs\") pod \"odh-model-controller-7f9f8cdc65-5dzzj\" (UID: \"ff6f5d92-c6a9-41d4-97f1-c646bd6291fe\") " pod="kserve/odh-model-controller-7f9f8cdc65-5dzzj"
Oct 30 22:04:38.742351 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:04:38.742319    6133 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h5gqk\" (UniqueName: \"kubernetes.io/projected/ff6f5d92-c6a9-41d4-97f1-c646bd6291fe-kube-api-access-h5gqk\") pod \"odh-model-controller-7f9f8cdc65-5dzzj\" (UID: \"ff6f5d92-c6a9-41d4-97f1-c646bd6291fe\") " pod="kserve/odh-model-controller-7f9f8cdc65-5dzzj"
Oct 30 22:04:38.875936 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:04:38.875849    6133 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/odh-model-controller-7f9f8cdc65-5dzzj"
Oct 30 22:04:39.177645 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:04:39.177569    6133 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kserve/odh-model-controller-7f9f8cdc65-5dzzj"]
Oct 30 22:04:39.180973 ip-10-0-143-246 kubenswrapper[6133]: W1030 22:04:39.180947    6133 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podff6f5d92_c6a9_41d4_97f1_c646bd6291fe.slice/crio-dd90b730cd5fb3a8f2705c38b334f598fd4b9bda25832ffcfc7f12412c60add3 WatchSource:0}: Error finding container dd90b730cd5fb3a8f2705c38b334f598fd4b9bda25832ffcfc7f12412c60add3: Status 404 returned error can't find the container with id dd90b730cd5fb3a8f2705c38b334f598fd4b9bda25832ffcfc7f12412c60add3
Oct 30 22:04:39.182273 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:04:39.182258    6133 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 22:04:39.420187 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:04:39.420157    6133 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kserve/odh-model-controller-7f9f8cdc65-5dzzj" event={"ID":"ff6f5d92-c6a9-41d4-97f1-c646bd6291fe","Type":"ContainerStarted","Data":"dd90b730cd5fb3a8f2705c38b334f598fd4b9bda25832ffcfc7f12412c60add3"}
Oct 30 22:04:39.526656 ip-10-0-143-246 kubenswrapper[6133]: E1030 22:04:39.526619    6133 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6jppj" containerName="extractor"
Oct 30 22:04:42.433289 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:04:42.433253    6133 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kserve/odh-model-controller-7f9f8cdc65-5dzzj" event={"ID":"ff6f5d92-c6a9-41d4-97f1-c646bd6291fe","Type":"ContainerStarted","Data":"cf7855074307f5c0eb722517f47e5a2f9d267ae1fffad6209ddb0d572ffbf0ce"}
Oct 30 22:04:42.433707 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:04:42.433363    6133 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/odh-model-controller-7f9f8cdc65-5dzzj"
Oct 30 22:04:42.474923 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:04:42.474873    6133 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kserve/odh-model-controller-7f9f8cdc65-5dzzj" podStartSLOduration=2.007983423 podStartE2EDuration="4.47485782s" totalImagesPullingTime="2.466874397s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:04:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:04:42.47475448 +0000 UTC m=+589.235510394" watchObservedRunningTime="2026-10-30 22:04:42.47485782 +0000 UTC m=+589.235613703"
Oct 30 22:04:53.438610 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:04:53.438582    6133 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/odh-model-controller-7f9f8cdc65-5dzzj"
Oct 30 22:04:54.871638 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:04:54.871599    6133 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kserve/s3-init-n2zmf"]
Oct 30 22:04:54.875373 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:04:54.875349    6133 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/s3-init-n2zmf"
Oct 30 22:04:54.890023 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:04:54.889982    6133 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"hf-token\""
Oct 30 22:04:54.890173 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:04:54.890027    6133 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"mlpipeline-s3-artifact\""
Oct 30 22:04:54.900154 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:04:54.900136    6133 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"default-dockercfg-xf6xv\""
Oct 30 22:04:54.903670 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:04:54.903644    6133 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kserve/s3-init-n2zmf"]
Oct 30 22:04:54.953592 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:04:54.953549    6133 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/61a1c92f-6d2c-40ca-a16d-7898749f9a19-model-cache\") pod \"s3-init-n2zmf\" (UID: \"61a1c92f-6d2c-40ca-a16d-7898749f9a19\") " pod="kserve/s3-init-n2zmf"
Oct 30 22:04:54.953592 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:04:54.953599    6133 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nrxpz\" (UniqueName: \"kubernetes.io/projected/61a1c92f-6d2c-40ca-a16d-7898749f9a19-kube-api-access-nrxpz\") pod \"s3-init-n2zmf\" (UID: \"61a1c92f-6d2c-40ca-a16d-7898749f9a19\") " pod="kserve/s3-init-n2zmf"
Oct 30 22:04:55.054060 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:04:55.054021    6133 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/61a1c92f-6d2c-40ca-a16d-7898749f9a19-model-cache\") pod \"s3-init-n2zmf\" (UID: \"61a1c92f-6d2c-40ca-a16d-7898749f9a19\") " pod="kserve/s3-init-n2zmf"
Oct 30 22:04:55.054060 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:04:55.054068    6133 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nrxpz\" (UniqueName: \"kubernetes.io/projected/61a1c92f-6d2c-40ca-a16d-7898749f9a19-kube-api-access-nrxpz\") pod \"s3-init-n2zmf\" (UID: \"61a1c92f-6d2c-40ca-a16d-7898749f9a19\") " pod="kserve/s3-init-n2zmf"
Oct 30 22:04:55.054442 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:04:55.054419    6133 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/61a1c92f-6d2c-40ca-a16d-7898749f9a19-model-cache\") pod \"s3-init-n2zmf\" (UID: \"61a1c92f-6d2c-40ca-a16d-7898749f9a19\") " pod="kserve/s3-init-n2zmf"
Oct 30 22:04:55.074777 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:04:55.074742    6133 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nrxpz\" (UniqueName: \"kubernetes.io/projected/61a1c92f-6d2c-40ca-a16d-7898749f9a19-kube-api-access-nrxpz\") pod \"s3-init-n2zmf\" (UID: \"61a1c92f-6d2c-40ca-a16d-7898749f9a19\") " pod="kserve/s3-init-n2zmf"
Oct 30 22:04:55.205147 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:04:55.205044    6133 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/s3-init-n2zmf"
Oct 30 22:04:55.540380 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:04:55.540347    6133 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kserve/s3-init-n2zmf"]
Oct 30 22:04:55.543872 ip-10-0-143-246 kubenswrapper[6133]: W1030 22:04:55.543844    6133 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod61a1c92f_6d2c_40ca_a16d_7898749f9a19.slice/crio-406a7aa93a7d70addb51205fdee3e654b178aec2680efebfd5d9a3a7db6f904a WatchSource:0}: Error finding container 406a7aa93a7d70addb51205fdee3e654b178aec2680efebfd5d9a3a7db6f904a: Status 404 returned error can't find the container with id 406a7aa93a7d70addb51205fdee3e654b178aec2680efebfd5d9a3a7db6f904a
Oct 30 22:04:56.487780 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:04:56.487745    6133 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kserve/s3-init-n2zmf" event={"ID":"61a1c92f-6d2c-40ca-a16d-7898749f9a19","Type":"ContainerStarted","Data":"406a7aa93a7d70addb51205fdee3e654b178aec2680efebfd5d9a3a7db6f904a"}
Oct 30 22:04:59.502704 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:04:59.502669    6133 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kserve/s3-init-n2zmf" event={"ID":"61a1c92f-6d2c-40ca-a16d-7898749f9a19","Type":"ContainerStarted","Data":"16280a36ebddc9d7236de074bcd6d2ddd126a86956769d8c027578da024b0d5a"}
Oct 30 22:05:00.507663 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:05:00.507633    6133 generic.go:356] "Generic (PLEG): container finished" podID="61a1c92f-6d2c-40ca-a16d-7898749f9a19" containerID="16280a36ebddc9d7236de074bcd6d2ddd126a86956769d8c027578da024b0d5a" exitCode=0
Oct 30 22:05:00.508076 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:05:00.507688    6133 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kserve/s3-init-n2zmf" event={"ID":"61a1c92f-6d2c-40ca-a16d-7898749f9a19","Type":"ContainerDied","Data":"16280a36ebddc9d7236de074bcd6d2ddd126a86956769d8c027578da024b0d5a"}
Oct 30 22:05:05.529251 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:05:05.529202    6133 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kserve/s3-init-n2zmf" event={"ID":"61a1c92f-6d2c-40ca-a16d-7898749f9a19","Type":"ContainerStarted","Data":"d1e61d472edfb56f646d80d852e0ff0d504a966d36c3be736276af7982b53fea"}
Oct 30 22:05:05.585856 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:05:05.585735    6133 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kserve/s3-init-n2zmf" podStartSLOduration=2.229963168 podStartE2EDuration="11.585721087s" totalImagesPullingTime="8.27774471s" totalInitContainerRuntime="1.078013209s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:04:54 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:05:05.585052826 +0000 UTC m=+612.345808715" watchObservedRunningTime="2026-10-30 22:05:05.585721087 +0000 UTC m=+612.346476976"
Oct 30 22:05:09.531736 ip-10-0-143-246 kubenswrapper[6133]: E1030 22:05:09.531689    6133 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6jppj" containerName="extractor"
Oct 30 22:05:39.527438 ip-10-0-143-246 kubenswrapper[6133]: E1030 22:05:39.527354    6133 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6jppj" containerName="extractor"
Oct 30 22:06:09.527705 ip-10-0-143-246 kubenswrapper[6133]: E1030 22:06:09.527662    6133 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6jppj" containerName="extractor"
Oct 30 22:06:10.762752 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:06:10.762716    6133 generic.go:356] "Generic (PLEG): container finished" podID="61a1c92f-6d2c-40ca-a16d-7898749f9a19" containerID="d1e61d472edfb56f646d80d852e0ff0d504a966d36c3be736276af7982b53fea" exitCode=0
Oct 30 22:06:10.763156 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:06:10.762768    6133 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kserve/s3-init-n2zmf" event={"ID":"61a1c92f-6d2c-40ca-a16d-7898749f9a19","Type":"ContainerDied","Data":"d1e61d472edfb56f646d80d852e0ff0d504a966d36c3be736276af7982b53fea"}
Oct 30 22:06:11.891965 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:06:11.891939    6133 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve/s3-init-n2zmf"
Oct 30 22:06:11.989103 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:06:11.989064    6133 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/61a1c92f-6d2c-40ca-a16d-7898749f9a19-kube-api-access-nrxpz\" (UniqueName: \"kubernetes.io/projected/61a1c92f-6d2c-40ca-a16d-7898749f9a19-kube-api-access-nrxpz\") pod \"61a1c92f-6d2c-40ca-a16d-7898749f9a19\" (UID: \"61a1c92f-6d2c-40ca-a16d-7898749f9a19\") "
Oct 30 22:06:11.989284 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:06:11.989120    6133 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/61a1c92f-6d2c-40ca-a16d-7898749f9a19-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/61a1c92f-6d2c-40ca-a16d-7898749f9a19-model-cache\") pod \"61a1c92f-6d2c-40ca-a16d-7898749f9a19\" (UID: \"61a1c92f-6d2c-40ca-a16d-7898749f9a19\") "
Oct 30 22:06:11.991336 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:06:11.991307    6133 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/61a1c92f-6d2c-40ca-a16d-7898749f9a19-kube-api-access-nrxpz" pod "61a1c92f-6d2c-40ca-a16d-7898749f9a19" (UID: "61a1c92f-6d2c-40ca-a16d-7898749f9a19"). InnerVolumeSpecName "kube-api-access-nrxpz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:06:12.007492 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:06:12.007461    6133 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/61a1c92f-6d2c-40ca-a16d-7898749f9a19-model-cache" pod "61a1c92f-6d2c-40ca-a16d-7898749f9a19" (UID: "61a1c92f-6d2c-40ca-a16d-7898749f9a19"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:06:12.090610 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:06:12.090531    6133 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nrxpz\" (UniqueName: \"kubernetes.io/projected/61a1c92f-6d2c-40ca-a16d-7898749f9a19-kube-api-access-nrxpz\") on node \"ip-10-0-143-246.ec2.internal\" DevicePath \"\""
Oct 30 22:06:12.090610 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:06:12.090562    6133 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/61a1c92f-6d2c-40ca-a16d-7898749f9a19-model-cache\") on node \"ip-10-0-143-246.ec2.internal\" DevicePath \"\""
Oct 30 22:06:12.770703 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:06:12.770675    6133 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kserve/s3-init-n2zmf" event={"ID":"61a1c92f-6d2c-40ca-a16d-7898749f9a19","Type":"ContainerDied","Data":"406a7aa93a7d70addb51205fdee3e654b178aec2680efebfd5d9a3a7db6f904a"}
Oct 30 22:06:12.770703 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:06:12.770708    6133 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="406a7aa93a7d70addb51205fdee3e654b178aec2680efebfd5d9a3a7db6f904a"
Oct 30 22:06:12.770912 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:06:12.770711    6133 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve/s3-init-n2zmf"
Oct 30 22:06:30.748719 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:06:30.748682    6133 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-gw0-7b6759dd5d/isvc-raw-sklearn-batcher-f9cfa-predictor-57b5c6b7cb-ztjg5"]
Oct 30 22:06:30.749146 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:06:30.749043    6133 cpu_manager.go:414] "containerMap: removing container" podUID="61a1c92f-6d2c-40ca-a16d-7898749f9a19" containerName="s3-init"
Oct 30 22:06:30.749146 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:06:30.749055    6133 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="61a1c92f-6d2c-40ca-a16d-7898749f9a19" containerName="s3-init"
Oct 30 22:06:30.749146 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:06:30.749068    6133 cpu_manager.go:414] "containerMap: removing container" podUID="61a1c92f-6d2c-40ca-a16d-7898749f9a19" containerName="download-hf-model"
Oct 30 22:06:30.749146 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:06:30.749074    6133 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="61a1c92f-6d2c-40ca-a16d-7898749f9a19" containerName="download-hf-model"
Oct 30 22:06:30.749146 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:06:30.749125    6133 memory_manager.go:365] "RemoveStaleState removing state" podUID="61a1c92f-6d2c-40ca-a16d-7898749f9a19" containerName="s3-init"
Oct 30 22:06:30.751512 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:06:30.751495    6133 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-7b6759dd5d/isvc-raw-sklearn-batcher-f9cfa-predictor-57b5c6b7cb-ztjg5"
Oct 30 22:06:30.766569 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:06:30.766544    6133 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw0-7b6759dd5d\"/\"odh-kserve-custom-ca-bundle\""
Oct 30 22:06:30.766569 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:06:30.766550    6133 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw0-7b6759dd5d\"/\"kube-root-ca.crt\""
Oct 30 22:06:30.766723 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:06:30.766570    6133 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw0-7b6759dd5d\"/\"isvc-raw-sklearn-batcher-f9cfa-kube-rbac-proxy-sar-config\""
Oct 30 22:06:30.777414 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:06:30.777387    6133 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw0-7b6759dd5d\"/\"seaweedfs-s3-creds\""
Oct 30 22:06:30.777414 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:06:30.777387    6133 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw0-7b6759dd5d\"/\"default-dockercfg-4b7xh\""
Oct 30 22:06:30.777568 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:06:30.777443    6133 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw0-7b6759dd5d\"/\"openshift-service-ca.crt\""
Oct 30 22:06:30.777694 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:06:30.777680    6133 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw0-7b6759dd5d\"/\"isvc-raw-sklearn-batcher-f9cfa-predictor-serving-cert\""
Oct 30 22:06:30.778318 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:06:30.778299    6133 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw0-7b6759dd5d/isvc-raw-sklearn-batcher-f9cfa-predictor-57b5c6b7cb-ztjg5"]
Oct 30 22:06:30.848827 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:06:30.848791    6133 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/de778448-704c-4d67-84f4-c66dfb1abb14-kube-rbac-proxy-sar-config\") pod \"isvc-raw-sklearn-batcher-f9cfa-predictor-57b5c6b7cb-ztjg5\" (UID: \"de778448-704c-4d67-84f4-c66dfb1abb14\") " pod="e2e-gw0-7b6759dd5d/isvc-raw-sklearn-batcher-f9cfa-predictor-57b5c6b7cb-ztjg5"
Oct 30 22:06:30.848827 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:06:30.848832    6133 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/de778448-704c-4d67-84f4-c66dfb1abb14-proxy-tls\") pod \"isvc-raw-sklearn-batcher-f9cfa-predictor-57b5c6b7cb-ztjg5\" (UID: \"de778448-704c-4d67-84f4-c66dfb1abb14\") " pod="e2e-gw0-7b6759dd5d/isvc-raw-sklearn-batcher-f9cfa-predictor-57b5c6b7cb-ztjg5"
Oct 30 22:06:30.849129 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:06:30.848864    6133 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5sld6\" (UniqueName: \"kubernetes.io/projected/de778448-704c-4d67-84f4-c66dfb1abb14-kube-api-access-5sld6\") pod \"isvc-raw-sklearn-batcher-f9cfa-predictor-57b5c6b7cb-ztjg5\" (UID: \"de778448-704c-4d67-84f4-c66dfb1abb14\") " pod="e2e-gw0-7b6759dd5d/isvc-raw-sklearn-batcher-f9cfa-predictor-57b5c6b7cb-ztjg5"
Oct 30 22:06:30.849129 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:06:30.848931    6133 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/de778448-704c-4d67-84f4-c66dfb1abb14-cabundle-cert\") pod \"isvc-raw-sklearn-batcher-f9cfa-predictor-57b5c6b7cb-ztjg5\" (UID: \"de778448-704c-4d67-84f4-c66dfb1abb14\") " pod="e2e-gw0-7b6759dd5d/isvc-raw-sklearn-batcher-f9cfa-predictor-57b5c6b7cb-ztjg5"
Oct 30 22:06:30.849129 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:06:30.849039    6133 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/de778448-704c-4d67-84f4-c66dfb1abb14-kserve-provision-location\") pod \"isvc-raw-sklearn-batcher-f9cfa-predictor-57b5c6b7cb-ztjg5\" (UID: \"de778448-704c-4d67-84f4-c66dfb1abb14\") " pod="e2e-gw0-7b6759dd5d/isvc-raw-sklearn-batcher-f9cfa-predictor-57b5c6b7cb-ztjg5"
Oct 30 22:06:30.949777 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:06:30.949723    6133 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/de778448-704c-4d67-84f4-c66dfb1abb14-kserve-provision-location\") pod \"isvc-raw-sklearn-batcher-f9cfa-predictor-57b5c6b7cb-ztjg5\" (UID: \"de778448-704c-4d67-84f4-c66dfb1abb14\") " pod="e2e-gw0-7b6759dd5d/isvc-raw-sklearn-batcher-f9cfa-predictor-57b5c6b7cb-ztjg5"
Oct 30 22:06:30.949978 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:06:30.949831    6133 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/de778448-704c-4d67-84f4-c66dfb1abb14-kube-rbac-proxy-sar-config\") pod \"isvc-raw-sklearn-batcher-f9cfa-predictor-57b5c6b7cb-ztjg5\" (UID: \"de778448-704c-4d67-84f4-c66dfb1abb14\") " pod="e2e-gw0-7b6759dd5d/isvc-raw-sklearn-batcher-f9cfa-predictor-57b5c6b7cb-ztjg5"
Oct 30 22:06:30.949978 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:06:30.949879    6133 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/de778448-704c-4d67-84f4-c66dfb1abb14-proxy-tls\") pod \"isvc-raw-sklearn-batcher-f9cfa-predictor-57b5c6b7cb-ztjg5\" (UID: \"de778448-704c-4d67-84f4-c66dfb1abb14\") " pod="e2e-gw0-7b6759dd5d/isvc-raw-sklearn-batcher-f9cfa-predictor-57b5c6b7cb-ztjg5"
Oct 30 22:06:30.949978 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:06:30.949923    6133 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5sld6\" (UniqueName: \"kubernetes.io/projected/de778448-704c-4d67-84f4-c66dfb1abb14-kube-api-access-5sld6\") pod \"isvc-raw-sklearn-batcher-f9cfa-predictor-57b5c6b7cb-ztjg5\" (UID: \"de778448-704c-4d67-84f4-c66dfb1abb14\") " pod="e2e-gw0-7b6759dd5d/isvc-raw-sklearn-batcher-f9cfa-predictor-57b5c6b7cb-ztjg5"
Oct 30 22:06:30.949978 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:06:30.949940    6133 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/de778448-704c-4d67-84f4-c66dfb1abb14-cabundle-cert\") pod \"isvc-raw-sklearn-batcher-f9cfa-predictor-57b5c6b7cb-ztjg5\" (UID: \"de778448-704c-4d67-84f4-c66dfb1abb14\") " pod="e2e-gw0-7b6759dd5d/isvc-raw-sklearn-batcher-f9cfa-predictor-57b5c6b7cb-ztjg5"
Oct 30 22:06:30.950277 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:06:30.950251    6133 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/de778448-704c-4d67-84f4-c66dfb1abb14-kserve-provision-location\") pod \"isvc-raw-sklearn-batcher-f9cfa-predictor-57b5c6b7cb-ztjg5\" (UID: \"de778448-704c-4d67-84f4-c66dfb1abb14\") " pod="e2e-gw0-7b6759dd5d/isvc-raw-sklearn-batcher-f9cfa-predictor-57b5c6b7cb-ztjg5"
Oct 30 22:06:30.950574 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:06:30.950548    6133 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/de778448-704c-4d67-84f4-c66dfb1abb14-cabundle-cert\") pod \"isvc-raw-sklearn-batcher-f9cfa-predictor-57b5c6b7cb-ztjg5\" (UID: \"de778448-704c-4d67-84f4-c66dfb1abb14\") " pod="e2e-gw0-7b6759dd5d/isvc-raw-sklearn-batcher-f9cfa-predictor-57b5c6b7cb-ztjg5"
Oct 30 22:06:30.950682 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:06:30.950584    6133 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/de778448-704c-4d67-84f4-c66dfb1abb14-kube-rbac-proxy-sar-config\") pod \"isvc-raw-sklearn-batcher-f9cfa-predictor-57b5c6b7cb-ztjg5\" (UID: \"de778448-704c-4d67-84f4-c66dfb1abb14\") " pod="e2e-gw0-7b6759dd5d/isvc-raw-sklearn-batcher-f9cfa-predictor-57b5c6b7cb-ztjg5"
Oct 30 22:06:30.952294 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:06:30.952272    6133 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/de778448-704c-4d67-84f4-c66dfb1abb14-proxy-tls\") pod \"isvc-raw-sklearn-batcher-f9cfa-predictor-57b5c6b7cb-ztjg5\" (UID: \"de778448-704c-4d67-84f4-c66dfb1abb14\") " pod="e2e-gw0-7b6759dd5d/isvc-raw-sklearn-batcher-f9cfa-predictor-57b5c6b7cb-ztjg5"
Oct 30 22:06:30.970161 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:06:30.970124    6133 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5sld6\" (UniqueName: \"kubernetes.io/projected/de778448-704c-4d67-84f4-c66dfb1abb14-kube-api-access-5sld6\") pod \"isvc-raw-sklearn-batcher-f9cfa-predictor-57b5c6b7cb-ztjg5\" (UID: \"de778448-704c-4d67-84f4-c66dfb1abb14\") " pod="e2e-gw0-7b6759dd5d/isvc-raw-sklearn-batcher-f9cfa-predictor-57b5c6b7cb-ztjg5"
Oct 30 22:06:31.066093 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:06:31.065982    6133 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-7b6759dd5d/isvc-raw-sklearn-batcher-f9cfa-predictor-57b5c6b7cb-ztjg5"
Oct 30 22:06:31.154194 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:06:31.154165    6133 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-gw1-2e5a75ca98/isvc-canary-rollback-v2-predictor-5f45d9c89b-tgdq4"]
Oct 30 22:06:31.157020 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:06:31.156983    6133 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-2e5a75ca98/isvc-canary-rollback-v2-predictor-5f45d9c89b-tgdq4"
Oct 30 22:06:31.171645 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:06:31.171618    6133 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw1-2e5a75ca98\"/\"isvc-canary-rollback-kube-rbac-proxy-sar-config\""
Oct 30 22:06:31.171645 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:06:31.171637    6133 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw1-2e5a75ca98\"/\"kube-root-ca.crt\""
Oct 30 22:06:31.174968 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:06:31.174948    6133 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw1-2e5a75ca98\"/\"openshift-service-ca.crt\""
Oct 30 22:06:31.175789 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:06:31.175772    6133 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw1-2e5a75ca98\"/\"default-dockercfg-d9bw2\""
Oct 30 22:06:31.176035 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:06:31.176015    6133 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw1-2e5a75ca98\"/\"seaweedfs-s3-creds\""
Oct 30 22:06:31.176035 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:06:31.176027    6133 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw1-2e5a75ca98\"/\"isvc-canary-rollback-v2-predictor-serving-cert\""
Oct 30 22:06:31.184614 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:06:31.184588    6133 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw1-2e5a75ca98/isvc-canary-rollback-v2-predictor-5f45d9c89b-tgdq4"]
Oct 30 22:06:31.253173 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:06:31.253136    6133 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/db99ab82-2dd5-4af2-9a65-ed086e65ff00-kube-rbac-proxy-sar-config\") pod \"isvc-canary-rollback-v2-predictor-5f45d9c89b-tgdq4\" (UID: \"db99ab82-2dd5-4af2-9a65-ed086e65ff00\") " pod="e2e-gw1-2e5a75ca98/isvc-canary-rollback-v2-predictor-5f45d9c89b-tgdq4"
Oct 30 22:06:31.253349 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:06:31.253186    6133 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/db99ab82-2dd5-4af2-9a65-ed086e65ff00-proxy-tls\") pod \"isvc-canary-rollback-v2-predictor-5f45d9c89b-tgdq4\" (UID: \"db99ab82-2dd5-4af2-9a65-ed086e65ff00\") " pod="e2e-gw1-2e5a75ca98/isvc-canary-rollback-v2-predictor-5f45d9c89b-tgdq4"
Oct 30 22:06:31.253349 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:06:31.253264    6133 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/db99ab82-2dd5-4af2-9a65-ed086e65ff00-cabundle-cert\") pod \"isvc-canary-rollback-v2-predictor-5f45d9c89b-tgdq4\" (UID: \"db99ab82-2dd5-4af2-9a65-ed086e65ff00\") " pod="e2e-gw1-2e5a75ca98/isvc-canary-rollback-v2-predictor-5f45d9c89b-tgdq4"
Oct 30 22:06:31.253349 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:06:31.253312    6133 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/db99ab82-2dd5-4af2-9a65-ed086e65ff00-kserve-provision-location\") pod \"isvc-canary-rollback-v2-predictor-5f45d9c89b-tgdq4\" (UID: \"db99ab82-2dd5-4af2-9a65-ed086e65ff00\") " pod="e2e-gw1-2e5a75ca98/isvc-canary-rollback-v2-predictor-5f45d9c89b-tgdq4"
Oct 30 22:06:31.253349 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:06:31.253343    6133 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j2cnw\" (UniqueName: \"kubernetes.io/projected/db99ab82-2dd5-4af2-9a65-ed086e65ff00-kube-api-access-j2cnw\") pod \"isvc-canary-rollback-v2-predictor-5f45d9c89b-tgdq4\" (UID: \"db99ab82-2dd5-4af2-9a65-ed086e65ff00\") " pod="e2e-gw1-2e5a75ca98/isvc-canary-rollback-v2-predictor-5f45d9c89b-tgdq4"
Oct 30 22:06:31.272067 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:06:31.272042    6133 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw1-2e5a75ca98\"/\"odh-kserve-custom-ca-bundle\""
Oct 30 22:06:31.353780 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:06:31.353699    6133 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/db99ab82-2dd5-4af2-9a65-ed086e65ff00-kube-rbac-proxy-sar-config\") pod \"isvc-canary-rollback-v2-predictor-5f45d9c89b-tgdq4\" (UID: \"db99ab82-2dd5-4af2-9a65-ed086e65ff00\") " pod="e2e-gw1-2e5a75ca98/isvc-canary-rollback-v2-predictor-5f45d9c89b-tgdq4"
Oct 30 22:06:31.353780 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:06:31.353739    6133 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/db99ab82-2dd5-4af2-9a65-ed086e65ff00-proxy-tls\") pod \"isvc-canary-rollback-v2-predictor-5f45d9c89b-tgdq4\" (UID: \"db99ab82-2dd5-4af2-9a65-ed086e65ff00\") " pod="e2e-gw1-2e5a75ca98/isvc-canary-rollback-v2-predictor-5f45d9c89b-tgdq4"
Oct 30 22:06:31.353780 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:06:31.353775    6133 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/db99ab82-2dd5-4af2-9a65-ed086e65ff00-cabundle-cert\") pod \"isvc-canary-rollback-v2-predictor-5f45d9c89b-tgdq4\" (UID: \"db99ab82-2dd5-4af2-9a65-ed086e65ff00\") " pod="e2e-gw1-2e5a75ca98/isvc-canary-rollback-v2-predictor-5f45d9c89b-tgdq4"
Oct 30 22:06:31.354097 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:06:31.353804    6133 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/db99ab82-2dd5-4af2-9a65-ed086e65ff00-kserve-provision-location\") pod \"isvc-canary-rollback-v2-predictor-5f45d9c89b-tgdq4\" (UID: \"db99ab82-2dd5-4af2-9a65-ed086e65ff00\") " pod="e2e-gw1-2e5a75ca98/isvc-canary-rollback-v2-predictor-5f45d9c89b-tgdq4"
Oct 30 22:06:31.354097 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:06:31.353831    6133 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j2cnw\" (UniqueName: \"kubernetes.io/projected/db99ab82-2dd5-4af2-9a65-ed086e65ff00-kube-api-access-j2cnw\") pod \"isvc-canary-rollback-v2-predictor-5f45d9c89b-tgdq4\" (UID: \"db99ab82-2dd5-4af2-9a65-ed086e65ff00\") " pod="e2e-gw1-2e5a75ca98/isvc-canary-rollback-v2-predictor-5f45d9c89b-tgdq4"
Oct 30 22:06:31.354293 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:06:31.354266    6133 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/db99ab82-2dd5-4af2-9a65-ed086e65ff00-kserve-provision-location\") pod \"isvc-canary-rollback-v2-predictor-5f45d9c89b-tgdq4\" (UID: \"db99ab82-2dd5-4af2-9a65-ed086e65ff00\") " pod="e2e-gw1-2e5a75ca98/isvc-canary-rollback-v2-predictor-5f45d9c89b-tgdq4"
Oct 30 22:06:31.354460 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:06:31.354435    6133 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/db99ab82-2dd5-4af2-9a65-ed086e65ff00-kube-rbac-proxy-sar-config\") pod \"isvc-canary-rollback-v2-predictor-5f45d9c89b-tgdq4\" (UID: \"db99ab82-2dd5-4af2-9a65-ed086e65ff00\") " pod="e2e-gw1-2e5a75ca98/isvc-canary-rollback-v2-predictor-5f45d9c89b-tgdq4"
Oct 30 22:06:31.354577 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:06:31.354530    6133 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/db99ab82-2dd5-4af2-9a65-ed086e65ff00-cabundle-cert\") pod \"isvc-canary-rollback-v2-predictor-5f45d9c89b-tgdq4\" (UID: \"db99ab82-2dd5-4af2-9a65-ed086e65ff00\") " pod="e2e-gw1-2e5a75ca98/isvc-canary-rollback-v2-predictor-5f45d9c89b-tgdq4"
Oct 30 22:06:31.356407 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:06:31.356388    6133 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/db99ab82-2dd5-4af2-9a65-ed086e65ff00-proxy-tls\") pod \"isvc-canary-rollback-v2-predictor-5f45d9c89b-tgdq4\" (UID: \"db99ab82-2dd5-4af2-9a65-ed086e65ff00\") " pod="e2e-gw1-2e5a75ca98/isvc-canary-rollback-v2-predictor-5f45d9c89b-tgdq4"
Oct 30 22:06:31.374708 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:06:31.374677    6133 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw0-7b6759dd5d/isvc-raw-sklearn-batcher-f9cfa-predictor-57b5c6b7cb-ztjg5"]
Oct 30 22:06:31.377392 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:06:31.377368    6133 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j2cnw\" (UniqueName: \"kubernetes.io/projected/db99ab82-2dd5-4af2-9a65-ed086e65ff00-kube-api-access-j2cnw\") pod \"isvc-canary-rollback-v2-predictor-5f45d9c89b-tgdq4\" (UID: \"db99ab82-2dd5-4af2-9a65-ed086e65ff00\") " pod="e2e-gw1-2e5a75ca98/isvc-canary-rollback-v2-predictor-5f45d9c89b-tgdq4"
Oct 30 22:06:31.377959 ip-10-0-143-246 kubenswrapper[6133]: W1030 22:06:31.377933    6133 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podde778448_704c_4d67_84f4_c66dfb1abb14.slice/crio-39723f73f253b027143ef92706b51c79077827185122cf13911063ce5de7313f WatchSource:0}: Error finding container 39723f73f253b027143ef92706b51c79077827185122cf13911063ce5de7313f: Status 404 returned error can't find the container with id 39723f73f253b027143ef92706b51c79077827185122cf13911063ce5de7313f
Oct 30 22:06:31.463366 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:06:31.463336    6133 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-gw1-2e5a75ca98/isvc-canary-rollback-predictor-75dd485c97-hwjqk"]
Oct 30 22:06:31.465883 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:06:31.465862    6133 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-2e5a75ca98/isvc-canary-rollback-predictor-75dd485c97-hwjqk"
Oct 30 22:06:31.471930 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:06:31.471901    6133 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-2e5a75ca98/isvc-canary-rollback-v2-predictor-5f45d9c89b-tgdq4"
Oct 30 22:06:31.480625 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:06:31.480600    6133 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw1-2e5a75ca98\"/\"isvc-canary-rollback-predictor-serving-cert\""
Oct 30 22:06:31.488841 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:06:31.488817    6133 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw1-2e5a75ca98/isvc-canary-rollback-predictor-75dd485c97-hwjqk"]
Oct 30 22:06:31.556349 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:06:31.556314    6133 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-568rk\" (UniqueName: \"kubernetes.io/projected/de2fed1c-cb86-4057-8cd3-bb41d36a9664-kube-api-access-568rk\") pod \"isvc-canary-rollback-predictor-75dd485c97-hwjqk\" (UID: \"de2fed1c-cb86-4057-8cd3-bb41d36a9664\") " pod="e2e-gw1-2e5a75ca98/isvc-canary-rollback-predictor-75dd485c97-hwjqk"
Oct 30 22:06:31.556525 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:06:31.556361    6133 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/de2fed1c-cb86-4057-8cd3-bb41d36a9664-proxy-tls\") pod \"isvc-canary-rollback-predictor-75dd485c97-hwjqk\" (UID: \"de2fed1c-cb86-4057-8cd3-bb41d36a9664\") " pod="e2e-gw1-2e5a75ca98/isvc-canary-rollback-predictor-75dd485c97-hwjqk"
Oct 30 22:06:31.556525 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:06:31.556475    6133 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/de2fed1c-cb86-4057-8cd3-bb41d36a9664-kserve-provision-location\") pod \"isvc-canary-rollback-predictor-75dd485c97-hwjqk\" (UID: \"de2fed1c-cb86-4057-8cd3-bb41d36a9664\") " pod="e2e-gw1-2e5a75ca98/isvc-canary-rollback-predictor-75dd485c97-hwjqk"
Oct 30 22:06:31.556629 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:06:31.556532    6133 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/de2fed1c-cb86-4057-8cd3-bb41d36a9664-cabundle-cert\") pod \"isvc-canary-rollback-predictor-75dd485c97-hwjqk\" (UID: \"de2fed1c-cb86-4057-8cd3-bb41d36a9664\") " pod="e2e-gw1-2e5a75ca98/isvc-canary-rollback-predictor-75dd485c97-hwjqk"
Oct 30 22:06:31.556629 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:06:31.556583    6133 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/de2fed1c-cb86-4057-8cd3-bb41d36a9664-kube-rbac-proxy-sar-config\") pod \"isvc-canary-rollback-predictor-75dd485c97-hwjqk\" (UID: \"de2fed1c-cb86-4057-8cd3-bb41d36a9664\") " pod="e2e-gw1-2e5a75ca98/isvc-canary-rollback-predictor-75dd485c97-hwjqk"
Oct 30 22:06:31.657306 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:06:31.657219    6133 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-568rk\" (UniqueName: \"kubernetes.io/projected/de2fed1c-cb86-4057-8cd3-bb41d36a9664-kube-api-access-568rk\") pod \"isvc-canary-rollback-predictor-75dd485c97-hwjqk\" (UID: \"de2fed1c-cb86-4057-8cd3-bb41d36a9664\") " pod="e2e-gw1-2e5a75ca98/isvc-canary-rollback-predictor-75dd485c97-hwjqk"
Oct 30 22:06:31.657306 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:06:31.657264    6133 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/de2fed1c-cb86-4057-8cd3-bb41d36a9664-proxy-tls\") pod \"isvc-canary-rollback-predictor-75dd485c97-hwjqk\" (UID: \"de2fed1c-cb86-4057-8cd3-bb41d36a9664\") " pod="e2e-gw1-2e5a75ca98/isvc-canary-rollback-predictor-75dd485c97-hwjqk"
Oct 30 22:06:31.657306 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:06:31.657304    6133 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/de2fed1c-cb86-4057-8cd3-bb41d36a9664-kserve-provision-location\") pod \"isvc-canary-rollback-predictor-75dd485c97-hwjqk\" (UID: \"de2fed1c-cb86-4057-8cd3-bb41d36a9664\") " pod="e2e-gw1-2e5a75ca98/isvc-canary-rollback-predictor-75dd485c97-hwjqk"
Oct 30 22:06:31.657576 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:06:31.657343    6133 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/de2fed1c-cb86-4057-8cd3-bb41d36a9664-cabundle-cert\") pod \"isvc-canary-rollback-predictor-75dd485c97-hwjqk\" (UID: \"de2fed1c-cb86-4057-8cd3-bb41d36a9664\") " pod="e2e-gw1-2e5a75ca98/isvc-canary-rollback-predictor-75dd485c97-hwjqk"
Oct 30 22:06:31.657576 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:06:31.657376    6133 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/de2fed1c-cb86-4057-8cd3-bb41d36a9664-kube-rbac-proxy-sar-config\") pod \"isvc-canary-rollback-predictor-75dd485c97-hwjqk\" (UID: \"de2fed1c-cb86-4057-8cd3-bb41d36a9664\") " pod="e2e-gw1-2e5a75ca98/isvc-canary-rollback-predictor-75dd485c97-hwjqk"
Oct 30 22:06:31.657760 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:06:31.657736    6133 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/de2fed1c-cb86-4057-8cd3-bb41d36a9664-kserve-provision-location\") pod \"isvc-canary-rollback-predictor-75dd485c97-hwjqk\" (UID: \"de2fed1c-cb86-4057-8cd3-bb41d36a9664\") " pod="e2e-gw1-2e5a75ca98/isvc-canary-rollback-predictor-75dd485c97-hwjqk"
Oct 30 22:06:31.658114 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:06:31.658091    6133 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/de2fed1c-cb86-4057-8cd3-bb41d36a9664-cabundle-cert\") pod \"isvc-canary-rollback-predictor-75dd485c97-hwjqk\" (UID: \"de2fed1c-cb86-4057-8cd3-bb41d36a9664\") " pod="e2e-gw1-2e5a75ca98/isvc-canary-rollback-predictor-75dd485c97-hwjqk"
Oct 30 22:06:31.658203 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:06:31.658117    6133 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/de2fed1c-cb86-4057-8cd3-bb41d36a9664-kube-rbac-proxy-sar-config\") pod \"isvc-canary-rollback-predictor-75dd485c97-hwjqk\" (UID: \"de2fed1c-cb86-4057-8cd3-bb41d36a9664\") " pod="e2e-gw1-2e5a75ca98/isvc-canary-rollback-predictor-75dd485c97-hwjqk"
Oct 30 22:06:31.659855 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:06:31.659823    6133 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/de2fed1c-cb86-4057-8cd3-bb41d36a9664-proxy-tls\") pod \"isvc-canary-rollback-predictor-75dd485c97-hwjqk\" (UID: \"de2fed1c-cb86-4057-8cd3-bb41d36a9664\") " pod="e2e-gw1-2e5a75ca98/isvc-canary-rollback-predictor-75dd485c97-hwjqk"
Oct 30 22:06:31.676516 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:06:31.676481    6133 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-568rk\" (UniqueName: \"kubernetes.io/projected/de2fed1c-cb86-4057-8cd3-bb41d36a9664-kube-api-access-568rk\") pod \"isvc-canary-rollback-predictor-75dd485c97-hwjqk\" (UID: \"de2fed1c-cb86-4057-8cd3-bb41d36a9664\") " pod="e2e-gw1-2e5a75ca98/isvc-canary-rollback-predictor-75dd485c97-hwjqk"
Oct 30 22:06:31.772853 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:06:31.772822    6133 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw1-2e5a75ca98/isvc-canary-rollback-v2-predictor-5f45d9c89b-tgdq4"]
Oct 30 22:06:31.775737 ip-10-0-143-246 kubenswrapper[6133]: W1030 22:06:31.775702    6133 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddb99ab82_2dd5_4af2_9a65_ed086e65ff00.slice/crio-4744899b425d0a7a521350474bec051add2955bd4758fbee8f6f5d752d77ff10 WatchSource:0}: Error finding container 4744899b425d0a7a521350474bec051add2955bd4758fbee8f6f5d752d77ff10: Status 404 returned error can't find the container with id 4744899b425d0a7a521350474bec051add2955bd4758fbee8f6f5d752d77ff10
Oct 30 22:06:31.783761 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:06:31.783729    6133 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-2e5a75ca98/isvc-canary-rollback-predictor-75dd485c97-hwjqk"
Oct 30 22:06:31.844118 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:06:31.841307    6133 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-7b6759dd5d/isvc-raw-sklearn-batcher-f9cfa-predictor-57b5c6b7cb-ztjg5" event={"ID":"de778448-704c-4d67-84f4-c66dfb1abb14","Type":"ContainerStarted","Data":"15e513e01fe357c2ad2da4c26c43cf92e9a53795467a9a1647ad301c9f968e3a"}
Oct 30 22:06:31.844118 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:06:31.841390    6133 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-7b6759dd5d/isvc-raw-sklearn-batcher-f9cfa-predictor-57b5c6b7cb-ztjg5" event={"ID":"de778448-704c-4d67-84f4-c66dfb1abb14","Type":"ContainerStarted","Data":"39723f73f253b027143ef92706b51c79077827185122cf13911063ce5de7313f"}
Oct 30 22:06:31.844458 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:06:31.844363    6133 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-2e5a75ca98/isvc-canary-rollback-v2-predictor-5f45d9c89b-tgdq4" event={"ID":"db99ab82-2dd5-4af2-9a65-ed086e65ff00","Type":"ContainerStarted","Data":"4744899b425d0a7a521350474bec051add2955bd4758fbee8f6f5d752d77ff10"}
Oct 30 22:06:32.082306 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:06:32.082274    6133 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw1-2e5a75ca98/isvc-canary-rollback-predictor-75dd485c97-hwjqk"]
Oct 30 22:06:32.086746 ip-10-0-143-246 kubenswrapper[6133]: W1030 22:06:32.086714    6133 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podde2fed1c_cb86_4057_8cd3_bb41d36a9664.slice/crio-8e7786426eb565a207954b1ee148ca5262087b3241e0dbc2687eabdc54c7a04f WatchSource:0}: Error finding container 8e7786426eb565a207954b1ee148ca5262087b3241e0dbc2687eabdc54c7a04f: Status 404 returned error can't find the container with id 8e7786426eb565a207954b1ee148ca5262087b3241e0dbc2687eabdc54c7a04f
Oct 30 22:06:32.849199 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:06:32.849160    6133 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-2e5a75ca98/isvc-canary-rollback-predictor-75dd485c97-hwjqk" event={"ID":"de2fed1c-cb86-4057-8cd3-bb41d36a9664","Type":"ContainerStarted","Data":"f33970db2601a9d2d31f74b66e6d54420c86e6b29bb21e98c0f963bdf6ceb542"}
Oct 30 22:06:32.849676 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:06:32.849271    6133 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-2e5a75ca98/isvc-canary-rollback-predictor-75dd485c97-hwjqk" event={"ID":"de2fed1c-cb86-4057-8cd3-bb41d36a9664","Type":"ContainerStarted","Data":"8e7786426eb565a207954b1ee148ca5262087b3241e0dbc2687eabdc54c7a04f"}
Oct 30 22:06:32.850678 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:06:32.850650    6133 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-2e5a75ca98/isvc-canary-rollback-v2-predictor-5f45d9c89b-tgdq4" event={"ID":"db99ab82-2dd5-4af2-9a65-ed086e65ff00","Type":"ContainerStarted","Data":"0d1b575a7eea5d35dad92698db35bf1d1f1d92394d80c07eddcc7a095ec09b91"}
Oct 30 22:06:35.862636 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:06:35.862600    6133 generic.go:356] "Generic (PLEG): container finished" podID="de778448-704c-4d67-84f4-c66dfb1abb14" containerID="15e513e01fe357c2ad2da4c26c43cf92e9a53795467a9a1647ad301c9f968e3a" exitCode=0
Oct 30 22:06:35.863154 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:06:35.862682    6133 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-7b6759dd5d/isvc-raw-sklearn-batcher-f9cfa-predictor-57b5c6b7cb-ztjg5" event={"ID":"de778448-704c-4d67-84f4-c66dfb1abb14","Type":"ContainerDied","Data":"15e513e01fe357c2ad2da4c26c43cf92e9a53795467a9a1647ad301c9f968e3a"}
Oct 30 22:06:35.864740 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:06:35.864711    6133 generic.go:356] "Generic (PLEG): container finished" podID="db99ab82-2dd5-4af2-9a65-ed086e65ff00" containerID="0d1b575a7eea5d35dad92698db35bf1d1f1d92394d80c07eddcc7a095ec09b91" exitCode=0
Oct 30 22:06:35.864870 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:06:35.864787    6133 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-2e5a75ca98/isvc-canary-rollback-v2-predictor-5f45d9c89b-tgdq4" event={"ID":"db99ab82-2dd5-4af2-9a65-ed086e65ff00","Type":"ContainerDied","Data":"0d1b575a7eea5d35dad92698db35bf1d1f1d92394d80c07eddcc7a095ec09b91"}
Oct 30 22:06:36.886803 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:06:36.885840    6133 generic.go:356] "Generic (PLEG): container finished" podID="de2fed1c-cb86-4057-8cd3-bb41d36a9664" containerID="f33970db2601a9d2d31f74b66e6d54420c86e6b29bb21e98c0f963bdf6ceb542" exitCode=0
Oct 30 22:06:36.886803 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:06:36.885891    6133 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-2e5a75ca98/isvc-canary-rollback-predictor-75dd485c97-hwjqk" event={"ID":"de2fed1c-cb86-4057-8cd3-bb41d36a9664","Type":"ContainerDied","Data":"f33970db2601a9d2d31f74b66e6d54420c86e6b29bb21e98c0f963bdf6ceb542"}
Oct 30 22:06:39.581076 ip-10-0-143-246 kubenswrapper[6133]: E1030 22:06:39.581032    6133 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6jppj" containerName="extractor"
Oct 30 22:07:04.106450 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:07:04.106413    6133 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-2e5a75ca98/isvc-canary-rollback-v2-predictor-5f45d9c89b-tgdq4" event={"ID":"db99ab82-2dd5-4af2-9a65-ed086e65ff00","Type":"ContainerStarted","Data":"3aa09d0aa80d1567295db8eb67a1478e0dff6e56fa6e0b033a62d0c213003085"}
Oct 30 22:07:05.116166 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:07:05.115888    6133 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-7b6759dd5d/isvc-raw-sklearn-batcher-f9cfa-predictor-57b5c6b7cb-ztjg5" event={"ID":"de778448-704c-4d67-84f4-c66dfb1abb14","Type":"ContainerStarted","Data":"1fd68968c7c8bc62402be1ef466c06389e3aec6ef8c140d2f52fdaa9a5a2a828"}
Oct 30 22:07:05.121142 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:07:05.121105    6133 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-2e5a75ca98/isvc-canary-rollback-predictor-75dd485c97-hwjqk" event={"ID":"de2fed1c-cb86-4057-8cd3-bb41d36a9664","Type":"ContainerStarted","Data":"8162df07f408bb1c9545ed5c3a4948c87f1f216c8294c0b5ba8f1a55dd1737c7"}
Oct 30 22:07:07.132024 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:07:07.131972    6133 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-7b6759dd5d/isvc-raw-sklearn-batcher-f9cfa-predictor-57b5c6b7cb-ztjg5" event={"ID":"de778448-704c-4d67-84f4-c66dfb1abb14","Type":"ContainerStarted","Data":"0a60bf4fb900b85fd05b56caf63f539b4ad305ffc1c9a1b550ef39c0bc85ea05"}
Oct 30 22:07:07.134020 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:07:07.133978    6133 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-2e5a75ca98/isvc-canary-rollback-predictor-75dd485c97-hwjqk" event={"ID":"de2fed1c-cb86-4057-8cd3-bb41d36a9664","Type":"ContainerStarted","Data":"36a4e2d6446ecbf2f4f1217a615fcec33103185a9a570eac77230fc900d75ff4"}
Oct 30 22:07:07.134168 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:07:07.134078    6133 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw1-2e5a75ca98/isvc-canary-rollback-predictor-75dd485c97-hwjqk"
Oct 30 22:07:07.134234 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:07:07.134197    6133 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw1-2e5a75ca98/isvc-canary-rollback-predictor-75dd485c97-hwjqk"
Oct 30 22:07:07.135866 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:07:07.135829    6133 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-2e5a75ca98/isvc-canary-rollback-predictor-75dd485c97-hwjqk" podUID="de2fed1c-cb86-4057-8cd3-bb41d36a9664" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.19:8080: connect: connection refused"
Oct 30 22:07:07.136282 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:07:07.136263    6133 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-2e5a75ca98/isvc-canary-rollback-v2-predictor-5f45d9c89b-tgdq4" event={"ID":"db99ab82-2dd5-4af2-9a65-ed086e65ff00","Type":"ContainerStarted","Data":"ac6d167373566d1cae1456a1f34847d197fd84c9b09d142e7d6b35ec5c3c5db4"}
Oct 30 22:07:07.136410 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:07:07.136394    6133 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw1-2e5a75ca98/isvc-canary-rollback-v2-predictor-5f45d9c89b-tgdq4"
Oct 30 22:07:07.136450 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:07:07.136443    6133 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw1-2e5a75ca98/isvc-canary-rollback-v2-predictor-5f45d9c89b-tgdq4"
Oct 30 22:07:07.137421 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:07:07.137396    6133 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-2e5a75ca98/isvc-canary-rollback-v2-predictor-5f45d9c89b-tgdq4" podUID="db99ab82-2dd5-4af2-9a65-ed086e65ff00" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.18:8080: connect: connection refused"
Oct 30 22:07:07.193068 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:07:07.193017    6133 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-gw1-2e5a75ca98/isvc-canary-rollback-predictor-75dd485c97-hwjqk" podStartSLOduration=2.365105339 podStartE2EDuration="36.192978168s" totalImagesPullingTime="30.060202403s" totalInitContainerRuntime="3.767670426s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:06:31 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:07:07.186264867 +0000 UTC m=+733.947020780" watchObservedRunningTime="2026-10-30 22:07:07.192978168 +0000 UTC m=+733.953734056"
Oct 30 22:07:07.228151 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:07:07.228099    6133 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-gw1-2e5a75ca98/isvc-canary-rollback-v2-predictor-5f45d9c89b-tgdq4" podStartSLOduration=1.561687154 podStartE2EDuration="36.228084501s" totalImagesPullingTime="30.782504042s" totalInitContainerRuntime="3.883893305s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:06:31 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:07:07.227274993 +0000 UTC m=+733.988030930" watchObservedRunningTime="2026-10-30 22:07:07.228084501 +0000 UTC m=+733.988840389"
Oct 30 22:07:08.140440 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:07:08.140392    6133 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-2e5a75ca98/isvc-canary-rollback-v2-predictor-5f45d9c89b-tgdq4" podUID="db99ab82-2dd5-4af2-9a65-ed086e65ff00" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.18:8080: connect: connection refused"
Oct 30 22:07:08.140917 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:07:08.140462    6133 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-2e5a75ca98/isvc-canary-rollback-predictor-75dd485c97-hwjqk" podUID="de2fed1c-cb86-4057-8cd3-bb41d36a9664" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.19:8080: connect: connection refused"
Oct 30 22:07:09.529197 ip-10-0-143-246 kubenswrapper[6133]: E1030 22:07:09.529161    6133 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6jppj" containerName="extractor"
Oct 30 22:07:10.151034 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:07:10.150974    6133 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-7b6759dd5d/isvc-raw-sklearn-batcher-f9cfa-predictor-57b5c6b7cb-ztjg5" event={"ID":"de778448-704c-4d67-84f4-c66dfb1abb14","Type":"ContainerStarted","Data":"215b67b68b881759f2ad3eedf42e532096c674e0e22d99bec8d6fd8b968be788"}
Oct 30 22:07:10.151288 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:07:10.151174    6133 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw0-7b6759dd5d/isvc-raw-sklearn-batcher-f9cfa-predictor-57b5c6b7cb-ztjg5"
Oct 30 22:07:10.218162 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:07:10.218109    6133 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-gw0-7b6759dd5d/isvc-raw-sklearn-batcher-f9cfa-predictor-57b5c6b7cb-ztjg5" podStartSLOduration=2.968240073 podStartE2EDuration="40.218095262s" totalImagesPullingTime="33.379903897s" totalInitContainerRuntime="3.869951292s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:06:30 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:07:10.217931864 +0000 UTC m=+736.978687753" watchObservedRunningTime="2026-10-30 22:07:10.218095262 +0000 UTC m=+736.978851163"
Oct 30 22:07:11.155391 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:07:11.155351    6133 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw0-7b6759dd5d/isvc-raw-sklearn-batcher-f9cfa-predictor-57b5c6b7cb-ztjg5"
Oct 30 22:07:11.155801 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:07:11.155437    6133 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw0-7b6759dd5d/isvc-raw-sklearn-batcher-f9cfa-predictor-57b5c6b7cb-ztjg5"
Oct 30 22:07:11.156901 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:07:11.156868    6133 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7b6759dd5d/isvc-raw-sklearn-batcher-f9cfa-predictor-57b5c6b7cb-ztjg5" podUID="de778448-704c-4d67-84f4-c66dfb1abb14" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.17:8080: connect: connection refused"
Oct 30 22:07:11.157510 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:07:11.157489    6133 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7b6759dd5d/isvc-raw-sklearn-batcher-f9cfa-predictor-57b5c6b7cb-ztjg5" podUID="de778448-704c-4d67-84f4-c66dfb1abb14" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 22:07:12.158858 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:07:12.158811    6133 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7b6759dd5d/isvc-raw-sklearn-batcher-f9cfa-predictor-57b5c6b7cb-ztjg5" podUID="de778448-704c-4d67-84f4-c66dfb1abb14" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.17:8080: connect: connection refused"
Oct 30 22:07:12.159347 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:07:12.159322    6133 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7b6759dd5d/isvc-raw-sklearn-batcher-f9cfa-predictor-57b5c6b7cb-ztjg5" podUID="de778448-704c-4d67-84f4-c66dfb1abb14" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 22:07:12.162969 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:07:12.162947    6133 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw0-7b6759dd5d/isvc-raw-sklearn-batcher-f9cfa-predictor-57b5c6b7cb-ztjg5"
Oct 30 22:07:13.144840 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:07:13.144813    6133 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw1-2e5a75ca98/isvc-canary-rollback-v2-predictor-5f45d9c89b-tgdq4"
Oct 30 22:07:13.145296 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:07:13.145280    6133 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw1-2e5a75ca98/isvc-canary-rollback-predictor-75dd485c97-hwjqk"
Oct 30 22:07:13.145357 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:07:13.145322    6133 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-2e5a75ca98/isvc-canary-rollback-v2-predictor-5f45d9c89b-tgdq4" podUID="db99ab82-2dd5-4af2-9a65-ed086e65ff00" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.18:8080: connect: connection refused"
Oct 30 22:07:13.145737 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:07:13.145719    6133 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-2e5a75ca98/isvc-canary-rollback-predictor-75dd485c97-hwjqk" podUID="de2fed1c-cb86-4057-8cd3-bb41d36a9664" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.19:8080: connect: connection refused"
Oct 30 22:07:13.162542 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:07:13.162503    6133 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7b6759dd5d/isvc-raw-sklearn-batcher-f9cfa-predictor-57b5c6b7cb-ztjg5" podUID="de778448-704c-4d67-84f4-c66dfb1abb14" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.17:8080: connect: connection refused"
Oct 30 22:07:13.162948 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:07:13.162826    6133 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7b6759dd5d/isvc-raw-sklearn-batcher-f9cfa-predictor-57b5c6b7cb-ztjg5" podUID="de778448-704c-4d67-84f4-c66dfb1abb14" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 22:07:23.145573 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:07:23.145531    6133 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-2e5a75ca98/isvc-canary-rollback-v2-predictor-5f45d9c89b-tgdq4" podUID="db99ab82-2dd5-4af2-9a65-ed086e65ff00" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.18:8080: connect: connection refused"
Oct 30 22:07:23.146074 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:07:23.145664    6133 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-2e5a75ca98/isvc-canary-rollback-predictor-75dd485c97-hwjqk" podUID="de2fed1c-cb86-4057-8cd3-bb41d36a9664" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.19:8080: connect: connection refused"
Oct 30 22:07:23.163078 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:07:23.163042    6133 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7b6759dd5d/isvc-raw-sklearn-batcher-f9cfa-predictor-57b5c6b7cb-ztjg5" podUID="de778448-704c-4d67-84f4-c66dfb1abb14" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.17:8080: connect: connection refused"
Oct 30 22:07:23.163555 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:07:23.163534    6133 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7b6759dd5d/isvc-raw-sklearn-batcher-f9cfa-predictor-57b5c6b7cb-ztjg5" podUID="de778448-704c-4d67-84f4-c66dfb1abb14" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 22:07:33.145329 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:07:33.145292    6133 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-2e5a75ca98/isvc-canary-rollback-v2-predictor-5f45d9c89b-tgdq4" podUID="db99ab82-2dd5-4af2-9a65-ed086e65ff00" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.18:8080: connect: connection refused"
Oct 30 22:07:33.145730 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:07:33.145690    6133 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-2e5a75ca98/isvc-canary-rollback-predictor-75dd485c97-hwjqk" podUID="de2fed1c-cb86-4057-8cd3-bb41d36a9664" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.19:8080: connect: connection refused"
Oct 30 22:07:33.163186 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:07:33.163147    6133 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7b6759dd5d/isvc-raw-sklearn-batcher-f9cfa-predictor-57b5c6b7cb-ztjg5" podUID="de778448-704c-4d67-84f4-c66dfb1abb14" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.17:8080: connect: connection refused"
Oct 30 22:07:33.163565 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:07:33.163545    6133 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7b6759dd5d/isvc-raw-sklearn-batcher-f9cfa-predictor-57b5c6b7cb-ztjg5" podUID="de778448-704c-4d67-84f4-c66dfb1abb14" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 22:07:39.528362 ip-10-0-143-246 kubenswrapper[6133]: E1030 22:07:39.528319    6133 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6jppj" containerName="extractor"
Oct 30 22:07:43.146027 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:07:43.145978    6133 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-2e5a75ca98/isvc-canary-rollback-predictor-75dd485c97-hwjqk" podUID="de2fed1c-cb86-4057-8cd3-bb41d36a9664" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.19:8080: connect: connection refused"
Oct 30 22:07:43.146436 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:07:43.145978    6133 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-2e5a75ca98/isvc-canary-rollback-v2-predictor-5f45d9c89b-tgdq4" podUID="db99ab82-2dd5-4af2-9a65-ed086e65ff00" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.18:8080: connect: connection refused"
Oct 30 22:07:43.163213 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:07:43.163173    6133 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7b6759dd5d/isvc-raw-sklearn-batcher-f9cfa-predictor-57b5c6b7cb-ztjg5" podUID="de778448-704c-4d67-84f4-c66dfb1abb14" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.17:8080: connect: connection refused"
Oct 30 22:07:43.163630 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:07:43.163608    6133 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7b6759dd5d/isvc-raw-sklearn-batcher-f9cfa-predictor-57b5c6b7cb-ztjg5" podUID="de778448-704c-4d67-84f4-c66dfb1abb14" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 22:07:53.145725 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:07:53.145684    6133 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-2e5a75ca98/isvc-canary-rollback-predictor-75dd485c97-hwjqk" podUID="de2fed1c-cb86-4057-8cd3-bb41d36a9664" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.19:8080: connect: connection refused"
Oct 30 22:07:53.146129 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:07:53.145684    6133 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-2e5a75ca98/isvc-canary-rollback-v2-predictor-5f45d9c89b-tgdq4" podUID="db99ab82-2dd5-4af2-9a65-ed086e65ff00" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.18:8080: connect: connection refused"
Oct 30 22:07:53.163202 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:07:53.163154    6133 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7b6759dd5d/isvc-raw-sklearn-batcher-f9cfa-predictor-57b5c6b7cb-ztjg5" podUID="de778448-704c-4d67-84f4-c66dfb1abb14" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.17:8080: connect: connection refused"
Oct 30 22:07:53.163622 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:07:53.163598    6133 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7b6759dd5d/isvc-raw-sklearn-batcher-f9cfa-predictor-57b5c6b7cb-ztjg5" podUID="de778448-704c-4d67-84f4-c66dfb1abb14" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 22:08:03.146118 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:08:03.146078    6133 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-2e5a75ca98/isvc-canary-rollback-predictor-75dd485c97-hwjqk" podUID="de2fed1c-cb86-4057-8cd3-bb41d36a9664" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.19:8080: connect: connection refused"
Oct 30 22:08:03.146548 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:08:03.146078    6133 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-2e5a75ca98/isvc-canary-rollback-v2-predictor-5f45d9c89b-tgdq4" podUID="db99ab82-2dd5-4af2-9a65-ed086e65ff00" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.18:8080: connect: connection refused"
Oct 30 22:08:03.162548 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:08:03.162499    6133 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7b6759dd5d/isvc-raw-sklearn-batcher-f9cfa-predictor-57b5c6b7cb-ztjg5" podUID="de778448-704c-4d67-84f4-c66dfb1abb14" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.17:8080: connect: connection refused"
Oct 30 22:08:03.163024 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:08:03.162977    6133 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7b6759dd5d/isvc-raw-sklearn-batcher-f9cfa-predictor-57b5c6b7cb-ztjg5" podUID="de778448-704c-4d67-84f4-c66dfb1abb14" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 22:08:09.527887 ip-10-0-143-246 kubenswrapper[6133]: E1030 22:08:09.527838    6133 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6jppj" containerName="extractor"
Oct 30 22:08:13.145829 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:08:13.145787    6133 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-2e5a75ca98/isvc-canary-rollback-predictor-75dd485c97-hwjqk" podUID="de2fed1c-cb86-4057-8cd3-bb41d36a9664" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.19:8080: connect: connection refused"
Oct 30 22:08:13.146327 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:08:13.145789    6133 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-2e5a75ca98/isvc-canary-rollback-v2-predictor-5f45d9c89b-tgdq4" podUID="db99ab82-2dd5-4af2-9a65-ed086e65ff00" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.18:8080: connect: connection refused"
Oct 30 22:08:13.163254 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:08:13.163208    6133 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7b6759dd5d/isvc-raw-sklearn-batcher-f9cfa-predictor-57b5c6b7cb-ztjg5" podUID="de778448-704c-4d67-84f4-c66dfb1abb14" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.17:8080: connect: connection refused"
Oct 30 22:08:13.163624 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:08:13.163597    6133 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7b6759dd5d/isvc-raw-sklearn-batcher-f9cfa-predictor-57b5c6b7cb-ztjg5" podUID="de778448-704c-4d67-84f4-c66dfb1abb14" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 22:08:23.145301 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:08:23.145227    6133 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-2e5a75ca98/isvc-canary-rollback-v2-predictor-5f45d9c89b-tgdq4" podUID="db99ab82-2dd5-4af2-9a65-ed086e65ff00" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.18:8080: connect: connection refused"
Oct 30 22:08:23.145771 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:08:23.145706    6133 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-2e5a75ca98/isvc-canary-rollback-predictor-75dd485c97-hwjqk" podUID="de2fed1c-cb86-4057-8cd3-bb41d36a9664" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.19:8080: connect: connection refused"
Oct 30 22:08:23.163216 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:08:23.163172    6133 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7b6759dd5d/isvc-raw-sklearn-batcher-f9cfa-predictor-57b5c6b7cb-ztjg5" podUID="de778448-704c-4d67-84f4-c66dfb1abb14" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.17:8080: connect: connection refused"
Oct 30 22:08:23.163664 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:08:23.163640    6133 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7b6759dd5d/isvc-raw-sklearn-batcher-f9cfa-predictor-57b5c6b7cb-ztjg5" podUID="de778448-704c-4d67-84f4-c66dfb1abb14" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 22:08:27.350950 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:08:27.350903    6133 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7b6759dd5d/isvc-raw-sklearn-batcher-f9cfa-predictor-57b5c6b7cb-ztjg5" podUID="de778448-704c-4d67-84f4-c66dfb1abb14" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.17:8080: connect: connection refused"
Oct 30 22:08:27.351595 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:08:27.351557    6133 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7b6759dd5d/isvc-raw-sklearn-batcher-f9cfa-predictor-57b5c6b7cb-ztjg5" podUID="de778448-704c-4d67-84f4-c66dfb1abb14" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 22:08:33.146155 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:08:33.146120    6133 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw1-2e5a75ca98/isvc-canary-rollback-v2-predictor-5f45d9c89b-tgdq4"
Oct 30 22:08:33.146653 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:08:33.146229    6133 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw1-2e5a75ca98/isvc-canary-rollback-predictor-75dd485c97-hwjqk"
Oct 30 22:08:33.584868 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:08:33.584829    6133 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw1-2e5a75ca98/isvc-canary-rollback-predictor-75dd485c97-hwjqk"]
Oct 30 22:08:33.585136 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:08:33.585094    6133 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw1-2e5a75ca98/isvc-canary-rollback-predictor-75dd485c97-hwjqk" podUID="de2fed1c-cb86-4057-8cd3-bb41d36a9664" containerName="kserve-container" containerID="cri-o://8162df07f408bb1c9545ed5c3a4948c87f1f216c8294c0b5ba8f1a55dd1737c7" gracePeriod=30
Oct 30 22:08:33.585201 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:08:33.585144    6133 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw1-2e5a75ca98/isvc-canary-rollback-predictor-75dd485c97-hwjqk" podUID="de2fed1c-cb86-4057-8cd3-bb41d36a9664" containerName="kube-rbac-proxy" containerID="cri-o://36a4e2d6446ecbf2f4f1217a615fcec33103185a9a570eac77230fc900d75ff4" gracePeriod=30
Oct 30 22:08:34.474907 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:08:34.474875    6133 generic.go:356] "Generic (PLEG): container finished" podID="de2fed1c-cb86-4057-8cd3-bb41d36a9664" containerID="36a4e2d6446ecbf2f4f1217a615fcec33103185a9a570eac77230fc900d75ff4" exitCode=0
Oct 30 22:08:34.475351 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:08:34.474924    6133 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-2e5a75ca98/isvc-canary-rollback-predictor-75dd485c97-hwjqk" event={"ID":"de2fed1c-cb86-4057-8cd3-bb41d36a9664","Type":"ContainerDied","Data":"36a4e2d6446ecbf2f4f1217a615fcec33103185a9a570eac77230fc900d75ff4"}
Oct 30 22:08:37.355568 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:08:37.355489    6133 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw0-7b6759dd5d/isvc-raw-sklearn-batcher-f9cfa-predictor-57b5c6b7cb-ztjg5"
Oct 30 22:08:37.356026 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:08:37.355593    6133 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw0-7b6759dd5d/isvc-raw-sklearn-batcher-f9cfa-predictor-57b5c6b7cb-ztjg5"
Oct 30 22:08:38.141273 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:08:38.141226    6133 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-2e5a75ca98/isvc-canary-rollback-predictor-75dd485c97-hwjqk" podUID="de2fed1c-cb86-4057-8cd3-bb41d36a9664" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.19:8643/healthz\": dial tcp 10.134.0.19:8643: connect: connection refused"
Oct 30 22:08:39.029210 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:08:39.029181    6133 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-2e5a75ca98/isvc-canary-rollback-predictor-75dd485c97-hwjqk"
Oct 30 22:08:39.125914 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:08:39.125825    6133 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/de2fed1c-cb86-4057-8cd3-bb41d36a9664-kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/de2fed1c-cb86-4057-8cd3-bb41d36a9664-kserve-provision-location\") pod \"de2fed1c-cb86-4057-8cd3-bb41d36a9664\" (UID: \"de2fed1c-cb86-4057-8cd3-bb41d36a9664\") "
Oct 30 22:08:39.125914 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:08:39.125892    6133 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/de2fed1c-cb86-4057-8cd3-bb41d36a9664-kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/de2fed1c-cb86-4057-8cd3-bb41d36a9664-kube-rbac-proxy-sar-config\") pod \"de2fed1c-cb86-4057-8cd3-bb41d36a9664\" (UID: \"de2fed1c-cb86-4057-8cd3-bb41d36a9664\") "
Oct 30 22:08:39.126185 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:08:39.125923    6133 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/de2fed1c-cb86-4057-8cd3-bb41d36a9664-proxy-tls\" (UniqueName: \"kubernetes.io/secret/de2fed1c-cb86-4057-8cd3-bb41d36a9664-proxy-tls\") pod \"de2fed1c-cb86-4057-8cd3-bb41d36a9664\" (UID: \"de2fed1c-cb86-4057-8cd3-bb41d36a9664\") "
Oct 30 22:08:39.126185 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:08:39.125949    6133 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/de2fed1c-cb86-4057-8cd3-bb41d36a9664-cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/de2fed1c-cb86-4057-8cd3-bb41d36a9664-cabundle-cert\") pod \"de2fed1c-cb86-4057-8cd3-bb41d36a9664\" (UID: \"de2fed1c-cb86-4057-8cd3-bb41d36a9664\") "
Oct 30 22:08:39.126185 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:08:39.126020    6133 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/de2fed1c-cb86-4057-8cd3-bb41d36a9664-kube-api-access-568rk\" (UniqueName: \"kubernetes.io/projected/de2fed1c-cb86-4057-8cd3-bb41d36a9664-kube-api-access-568rk\") pod \"de2fed1c-cb86-4057-8cd3-bb41d36a9664\" (UID: \"de2fed1c-cb86-4057-8cd3-bb41d36a9664\") "
Oct 30 22:08:39.126346 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:08:39.126297    6133 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/de2fed1c-cb86-4057-8cd3-bb41d36a9664-kserve-provision-location" pod "de2fed1c-cb86-4057-8cd3-bb41d36a9664" (UID: "de2fed1c-cb86-4057-8cd3-bb41d36a9664"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:08:39.126346 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:08:39.126319    6133 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/de2fed1c-cb86-4057-8cd3-bb41d36a9664-kube-rbac-proxy-sar-config" pod "de2fed1c-cb86-4057-8cd3-bb41d36a9664" (UID: "de2fed1c-cb86-4057-8cd3-bb41d36a9664"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:08:39.126438 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:08:39.126378    6133 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/de2fed1c-cb86-4057-8cd3-bb41d36a9664-cabundle-cert" pod "de2fed1c-cb86-4057-8cd3-bb41d36a9664" (UID: "de2fed1c-cb86-4057-8cd3-bb41d36a9664"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:08:39.128218 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:08:39.128194    6133 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/de2fed1c-cb86-4057-8cd3-bb41d36a9664-proxy-tls" pod "de2fed1c-cb86-4057-8cd3-bb41d36a9664" (UID: "de2fed1c-cb86-4057-8cd3-bb41d36a9664"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:08:39.128312 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:08:39.128262    6133 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/de2fed1c-cb86-4057-8cd3-bb41d36a9664-kube-api-access-568rk" pod "de2fed1c-cb86-4057-8cd3-bb41d36a9664" (UID: "de2fed1c-cb86-4057-8cd3-bb41d36a9664"). InnerVolumeSpecName "kube-api-access-568rk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:08:39.227273 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:08:39.227235    6133 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/de2fed1c-cb86-4057-8cd3-bb41d36a9664-kserve-provision-location\") on node \"ip-10-0-143-246.ec2.internal\" DevicePath \"\""
Oct 30 22:08:39.227273 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:08:39.227264    6133 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/de2fed1c-cb86-4057-8cd3-bb41d36a9664-kube-rbac-proxy-sar-config\") on node \"ip-10-0-143-246.ec2.internal\" DevicePath \"\""
Oct 30 22:08:39.227273 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:08:39.227275    6133 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/de2fed1c-cb86-4057-8cd3-bb41d36a9664-proxy-tls\") on node \"ip-10-0-143-246.ec2.internal\" DevicePath \"\""
Oct 30 22:08:39.227273 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:08:39.227284    6133 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/de2fed1c-cb86-4057-8cd3-bb41d36a9664-cabundle-cert\") on node \"ip-10-0-143-246.ec2.internal\" DevicePath \"\""
Oct 30 22:08:39.227549 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:08:39.227294    6133 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-568rk\" (UniqueName: \"kubernetes.io/projected/de2fed1c-cb86-4057-8cd3-bb41d36a9664-kube-api-access-568rk\") on node \"ip-10-0-143-246.ec2.internal\" DevicePath \"\""
Oct 30 22:08:39.496066 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:08:39.496030    6133 generic.go:356] "Generic (PLEG): container finished" podID="de2fed1c-cb86-4057-8cd3-bb41d36a9664" containerID="8162df07f408bb1c9545ed5c3a4948c87f1f216c8294c0b5ba8f1a55dd1737c7" exitCode=0
Oct 30 22:08:39.496233 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:08:39.496101    6133 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-2e5a75ca98/isvc-canary-rollback-predictor-75dd485c97-hwjqk"
Oct 30 22:08:39.496233 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:08:39.496113    6133 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-2e5a75ca98/isvc-canary-rollback-predictor-75dd485c97-hwjqk" event={"ID":"de2fed1c-cb86-4057-8cd3-bb41d36a9664","Type":"ContainerDied","Data":"8162df07f408bb1c9545ed5c3a4948c87f1f216c8294c0b5ba8f1a55dd1737c7"}
Oct 30 22:08:39.496233 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:08:39.496179    6133 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-2e5a75ca98/isvc-canary-rollback-predictor-75dd485c97-hwjqk" event={"ID":"de2fed1c-cb86-4057-8cd3-bb41d36a9664","Type":"ContainerDied","Data":"8e7786426eb565a207954b1ee148ca5262087b3241e0dbc2687eabdc54c7a04f"}
Oct 30 22:08:39.496233 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:08:39.496208    6133 scope.go:122] "RemoveContainer" containerID="36a4e2d6446ecbf2f4f1217a615fcec33103185a9a570eac77230fc900d75ff4"
Oct 30 22:08:39.504686 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:08:39.504667    6133 scope.go:122] "RemoveContainer" containerID="8162df07f408bb1c9545ed5c3a4948c87f1f216c8294c0b5ba8f1a55dd1737c7"
Oct 30 22:08:39.512920 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:08:39.512896    6133 scope.go:122] "RemoveContainer" containerID="f33970db2601a9d2d31f74b66e6d54420c86e6b29bb21e98c0f963bdf6ceb542"
Oct 30 22:08:39.521265 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:08:39.521237    6133 scope.go:122] "RemoveContainer" containerID="36a4e2d6446ecbf2f4f1217a615fcec33103185a9a570eac77230fc900d75ff4"
Oct 30 22:08:39.521633 ip-10-0-143-246 kubenswrapper[6133]: E1030 22:08:39.521609    6133 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"36a4e2d6446ecbf2f4f1217a615fcec33103185a9a570eac77230fc900d75ff4\": container with ID starting with 36a4e2d6446ecbf2f4f1217a615fcec33103185a9a570eac77230fc900d75ff4 not found: ID does not exist" containerID="36a4e2d6446ecbf2f4f1217a615fcec33103185a9a570eac77230fc900d75ff4"
Oct 30 22:08:39.521718 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:08:39.521646    6133 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"36a4e2d6446ecbf2f4f1217a615fcec33103185a9a570eac77230fc900d75ff4"} err="failed to get container status \"36a4e2d6446ecbf2f4f1217a615fcec33103185a9a570eac77230fc900d75ff4\": rpc error: code = NotFound desc = could not find container \"36a4e2d6446ecbf2f4f1217a615fcec33103185a9a570eac77230fc900d75ff4\": container with ID starting with 36a4e2d6446ecbf2f4f1217a615fcec33103185a9a570eac77230fc900d75ff4 not found: ID does not exist"
Oct 30 22:08:39.521718 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:08:39.521666    6133 scope.go:122] "RemoveContainer" containerID="8162df07f408bb1c9545ed5c3a4948c87f1f216c8294c0b5ba8f1a55dd1737c7"
Oct 30 22:08:39.521975 ip-10-0-143-246 kubenswrapper[6133]: E1030 22:08:39.521954    6133 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8162df07f408bb1c9545ed5c3a4948c87f1f216c8294c0b5ba8f1a55dd1737c7\": container with ID starting with 8162df07f408bb1c9545ed5c3a4948c87f1f216c8294c0b5ba8f1a55dd1737c7 not found: ID does not exist" containerID="8162df07f408bb1c9545ed5c3a4948c87f1f216c8294c0b5ba8f1a55dd1737c7"
Oct 30 22:08:39.522085 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:08:39.521984    6133 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8162df07f408bb1c9545ed5c3a4948c87f1f216c8294c0b5ba8f1a55dd1737c7"} err="failed to get container status \"8162df07f408bb1c9545ed5c3a4948c87f1f216c8294c0b5ba8f1a55dd1737c7\": rpc error: code = NotFound desc = could not find container \"8162df07f408bb1c9545ed5c3a4948c87f1f216c8294c0b5ba8f1a55dd1737c7\": container with ID starting with 8162df07f408bb1c9545ed5c3a4948c87f1f216c8294c0b5ba8f1a55dd1737c7 not found: ID does not exist"
Oct 30 22:08:39.522085 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:08:39.522049    6133 scope.go:122] "RemoveContainer" containerID="f33970db2601a9d2d31f74b66e6d54420c86e6b29bb21e98c0f963bdf6ceb542"
Oct 30 22:08:39.522356 ip-10-0-143-246 kubenswrapper[6133]: E1030 22:08:39.522331    6133 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f33970db2601a9d2d31f74b66e6d54420c86e6b29bb21e98c0f963bdf6ceb542\": container with ID starting with f33970db2601a9d2d31f74b66e6d54420c86e6b29bb21e98c0f963bdf6ceb542 not found: ID does not exist" containerID="f33970db2601a9d2d31f74b66e6d54420c86e6b29bb21e98c0f963bdf6ceb542"
Oct 30 22:08:39.522402 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:08:39.522365    6133 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f33970db2601a9d2d31f74b66e6d54420c86e6b29bb21e98c0f963bdf6ceb542"} err="failed to get container status \"f33970db2601a9d2d31f74b66e6d54420c86e6b29bb21e98c0f963bdf6ceb542\": rpc error: code = NotFound desc = could not find container \"f33970db2601a9d2d31f74b66e6d54420c86e6b29bb21e98c0f963bdf6ceb542\": container with ID starting with f33970db2601a9d2d31f74b66e6d54420c86e6b29bb21e98c0f963bdf6ceb542 not found: ID does not exist"
Oct 30 22:08:39.529613 ip-10-0-143-246 kubenswrapper[6133]: E1030 22:08:39.529582    6133 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6jppj" containerName="extractor"
Oct 30 22:08:39.550730 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:08:39.550701    6133 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw1-2e5a75ca98/isvc-canary-rollback-predictor-75dd485c97-hwjqk"]
Oct 30 22:08:39.556071 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:08:39.556048    6133 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-gw1-2e5a75ca98/isvc-canary-rollback-predictor-75dd485c97-hwjqk"]
Oct 30 22:08:40.815161 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:08:40.815122    6133 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw1-2e5a75ca98/isvc-canary-rollback-v2-predictor-5f45d9c89b-tgdq4"]
Oct 30 22:08:40.815539 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:08:40.815350    6133 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw1-2e5a75ca98/isvc-canary-rollback-v2-predictor-5f45d9c89b-tgdq4" podUID="db99ab82-2dd5-4af2-9a65-ed086e65ff00" containerName="kserve-container" containerID="cri-o://3aa09d0aa80d1567295db8eb67a1478e0dff6e56fa6e0b033a62d0c213003085" gracePeriod=30
Oct 30 22:08:40.815539 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:08:40.815382    6133 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw1-2e5a75ca98/isvc-canary-rollback-v2-predictor-5f45d9c89b-tgdq4" podUID="db99ab82-2dd5-4af2-9a65-ed086e65ff00" containerName="kube-rbac-proxy" containerID="cri-o://ac6d167373566d1cae1456a1f34847d197fd84c9b09d142e7d6b35ec5c3c5db4" gracePeriod=30
Oct 30 22:08:40.998410 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:08:40.998379    6133 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-gw1-2e5a75ca98/isvc-canary-rollback-predictor-75dd485c97-g9nb8"]
Oct 30 22:08:40.998724 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:08:40.998710    6133 cpu_manager.go:414] "containerMap: removing container" podUID="de2fed1c-cb86-4057-8cd3-bb41d36a9664" containerName="kube-rbac-proxy"
Oct 30 22:08:40.998766 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:08:40.998725    6133 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="de2fed1c-cb86-4057-8cd3-bb41d36a9664" containerName="kube-rbac-proxy"
Oct 30 22:08:40.998766 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:08:40.998752    6133 cpu_manager.go:414] "containerMap: removing container" podUID="de2fed1c-cb86-4057-8cd3-bb41d36a9664" containerName="storage-initializer"
Oct 30 22:08:40.998766 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:08:40.998759    6133 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="de2fed1c-cb86-4057-8cd3-bb41d36a9664" containerName="storage-initializer"
Oct 30 22:08:40.998766 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:08:40.998766    6133 cpu_manager.go:414] "containerMap: removing container" podUID="de2fed1c-cb86-4057-8cd3-bb41d36a9664" containerName="kserve-container"
Oct 30 22:08:40.998891 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:08:40.998771    6133 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="de2fed1c-cb86-4057-8cd3-bb41d36a9664" containerName="kserve-container"
Oct 30 22:08:40.998891 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:08:40.998817    6133 memory_manager.go:365] "RemoveStaleState removing state" podUID="de2fed1c-cb86-4057-8cd3-bb41d36a9664" containerName="kserve-container"
Oct 30 22:08:40.998891 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:08:40.998828    6133 memory_manager.go:365] "RemoveStaleState removing state" podUID="de2fed1c-cb86-4057-8cd3-bb41d36a9664" containerName="kube-rbac-proxy"
Oct 30 22:08:41.001875 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:08:41.001841    6133 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-2e5a75ca98/isvc-canary-rollback-predictor-75dd485c97-g9nb8"
Oct 30 22:08:41.018066 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:08:41.018041    6133 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw1-2e5a75ca98\"/\"isvc-canary-rollback-predictor-serving-cert\""
Oct 30 22:08:41.023782 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:08:41.023749    6133 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw1-2e5a75ca98/isvc-canary-rollback-predictor-75dd485c97-g9nb8"]
Oct 30 22:08:41.145422 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:08:41.145320    6133 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/6257bde3-8c42-42ab-ab0c-b74521d57bb0-kube-rbac-proxy-sar-config\") pod \"isvc-canary-rollback-predictor-75dd485c97-g9nb8\" (UID: \"6257bde3-8c42-42ab-ab0c-b74521d57bb0\") " pod="e2e-gw1-2e5a75ca98/isvc-canary-rollback-predictor-75dd485c97-g9nb8"
Oct 30 22:08:41.145422 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:08:41.145398    6133 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/6257bde3-8c42-42ab-ab0c-b74521d57bb0-kserve-provision-location\") pod \"isvc-canary-rollback-predictor-75dd485c97-g9nb8\" (UID: \"6257bde3-8c42-42ab-ab0c-b74521d57bb0\") " pod="e2e-gw1-2e5a75ca98/isvc-canary-rollback-predictor-75dd485c97-g9nb8"
Oct 30 22:08:41.145661 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:08:41.145478    6133 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/6257bde3-8c42-42ab-ab0c-b74521d57bb0-proxy-tls\") pod \"isvc-canary-rollback-predictor-75dd485c97-g9nb8\" (UID: \"6257bde3-8c42-42ab-ab0c-b74521d57bb0\") " pod="e2e-gw1-2e5a75ca98/isvc-canary-rollback-predictor-75dd485c97-g9nb8"
Oct 30 22:08:41.145661 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:08:41.145518    6133 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dqz5n\" (UniqueName: \"kubernetes.io/projected/6257bde3-8c42-42ab-ab0c-b74521d57bb0-kube-api-access-dqz5n\") pod \"isvc-canary-rollback-predictor-75dd485c97-g9nb8\" (UID: \"6257bde3-8c42-42ab-ab0c-b74521d57bb0\") " pod="e2e-gw1-2e5a75ca98/isvc-canary-rollback-predictor-75dd485c97-g9nb8"
Oct 30 22:08:41.145661 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:08:41.145607    6133 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/6257bde3-8c42-42ab-ab0c-b74521d57bb0-cabundle-cert\") pod \"isvc-canary-rollback-predictor-75dd485c97-g9nb8\" (UID: \"6257bde3-8c42-42ab-ab0c-b74521d57bb0\") " pod="e2e-gw1-2e5a75ca98/isvc-canary-rollback-predictor-75dd485c97-g9nb8"
Oct 30 22:08:41.246221 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:08:41.246175    6133 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/6257bde3-8c42-42ab-ab0c-b74521d57bb0-kube-rbac-proxy-sar-config\") pod \"isvc-canary-rollback-predictor-75dd485c97-g9nb8\" (UID: \"6257bde3-8c42-42ab-ab0c-b74521d57bb0\") " pod="e2e-gw1-2e5a75ca98/isvc-canary-rollback-predictor-75dd485c97-g9nb8"
Oct 30 22:08:41.246438 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:08:41.246245    6133 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/6257bde3-8c42-42ab-ab0c-b74521d57bb0-kserve-provision-location\") pod \"isvc-canary-rollback-predictor-75dd485c97-g9nb8\" (UID: \"6257bde3-8c42-42ab-ab0c-b74521d57bb0\") " pod="e2e-gw1-2e5a75ca98/isvc-canary-rollback-predictor-75dd485c97-g9nb8"
Oct 30 22:08:41.246438 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:08:41.246264    6133 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/6257bde3-8c42-42ab-ab0c-b74521d57bb0-proxy-tls\") pod \"isvc-canary-rollback-predictor-75dd485c97-g9nb8\" (UID: \"6257bde3-8c42-42ab-ab0c-b74521d57bb0\") " pod="e2e-gw1-2e5a75ca98/isvc-canary-rollback-predictor-75dd485c97-g9nb8"
Oct 30 22:08:41.246438 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:08:41.246281    6133 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dqz5n\" (UniqueName: \"kubernetes.io/projected/6257bde3-8c42-42ab-ab0c-b74521d57bb0-kube-api-access-dqz5n\") pod \"isvc-canary-rollback-predictor-75dd485c97-g9nb8\" (UID: \"6257bde3-8c42-42ab-ab0c-b74521d57bb0\") " pod="e2e-gw1-2e5a75ca98/isvc-canary-rollback-predictor-75dd485c97-g9nb8"
Oct 30 22:08:41.246438 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:08:41.246309    6133 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/6257bde3-8c42-42ab-ab0c-b74521d57bb0-cabundle-cert\") pod \"isvc-canary-rollback-predictor-75dd485c97-g9nb8\" (UID: \"6257bde3-8c42-42ab-ab0c-b74521d57bb0\") " pod="e2e-gw1-2e5a75ca98/isvc-canary-rollback-predictor-75dd485c97-g9nb8"
Oct 30 22:08:41.246806 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:08:41.246772    6133 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/6257bde3-8c42-42ab-ab0c-b74521d57bb0-kserve-provision-location\") pod \"isvc-canary-rollback-predictor-75dd485c97-g9nb8\" (UID: \"6257bde3-8c42-42ab-ab0c-b74521d57bb0\") " pod="e2e-gw1-2e5a75ca98/isvc-canary-rollback-predictor-75dd485c97-g9nb8"
Oct 30 22:08:41.247017 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:08:41.246970    6133 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/6257bde3-8c42-42ab-ab0c-b74521d57bb0-kube-rbac-proxy-sar-config\") pod \"isvc-canary-rollback-predictor-75dd485c97-g9nb8\" (UID: \"6257bde3-8c42-42ab-ab0c-b74521d57bb0\") " pod="e2e-gw1-2e5a75ca98/isvc-canary-rollback-predictor-75dd485c97-g9nb8"
Oct 30 22:08:41.247083 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:08:41.247070    6133 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/6257bde3-8c42-42ab-ab0c-b74521d57bb0-cabundle-cert\") pod \"isvc-canary-rollback-predictor-75dd485c97-g9nb8\" (UID: \"6257bde3-8c42-42ab-ab0c-b74521d57bb0\") " pod="e2e-gw1-2e5a75ca98/isvc-canary-rollback-predictor-75dd485c97-g9nb8"
Oct 30 22:08:41.248703 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:08:41.248679    6133 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/6257bde3-8c42-42ab-ab0c-b74521d57bb0-proxy-tls\") pod \"isvc-canary-rollback-predictor-75dd485c97-g9nb8\" (UID: \"6257bde3-8c42-42ab-ab0c-b74521d57bb0\") " pod="e2e-gw1-2e5a75ca98/isvc-canary-rollback-predictor-75dd485c97-g9nb8"
Oct 30 22:08:41.268079 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:08:41.268053    6133 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dqz5n\" (UniqueName: \"kubernetes.io/projected/6257bde3-8c42-42ab-ab0c-b74521d57bb0-kube-api-access-dqz5n\") pod \"isvc-canary-rollback-predictor-75dd485c97-g9nb8\" (UID: \"6257bde3-8c42-42ab-ab0c-b74521d57bb0\") " pod="e2e-gw1-2e5a75ca98/isvc-canary-rollback-predictor-75dd485c97-g9nb8"
Oct 30 22:08:41.318959 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:08:41.318900    6133 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-2e5a75ca98/isvc-canary-rollback-predictor-75dd485c97-g9nb8"
Oct 30 22:08:41.356124 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:08:41.356088    6133 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="de2fed1c-cb86-4057-8cd3-bb41d36a9664" path="/var/lib/kubelet/pods/de2fed1c-cb86-4057-8cd3-bb41d36a9664/volumes"
Oct 30 22:08:41.508019 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:08:41.507972    6133 generic.go:356] "Generic (PLEG): container finished" podID="db99ab82-2dd5-4af2-9a65-ed086e65ff00" containerID="ac6d167373566d1cae1456a1f34847d197fd84c9b09d142e7d6b35ec5c3c5db4" exitCode=0
Oct 30 22:08:41.508205 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:08:41.508047    6133 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-2e5a75ca98/isvc-canary-rollback-v2-predictor-5f45d9c89b-tgdq4" event={"ID":"db99ab82-2dd5-4af2-9a65-ed086e65ff00","Type":"ContainerDied","Data":"ac6d167373566d1cae1456a1f34847d197fd84c9b09d142e7d6b35ec5c3c5db4"}
Oct 30 22:08:41.623744 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:08:41.623718    6133 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw1-2e5a75ca98/isvc-canary-rollback-predictor-75dd485c97-g9nb8"]
Oct 30 22:08:41.625617 ip-10-0-143-246 kubenswrapper[6133]: W1030 22:08:41.625589    6133 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6257bde3_8c42_42ab_ab0c_b74521d57bb0.slice/crio-6d08538cb2b4e5e5ca80a9e47ac3259718cdb25b537d5c2708eeddb7e76ae2af WatchSource:0}: Error finding container 6d08538cb2b4e5e5ca80a9e47ac3259718cdb25b537d5c2708eeddb7e76ae2af: Status 404 returned error can't find the container with id 6d08538cb2b4e5e5ca80a9e47ac3259718cdb25b537d5c2708eeddb7e76ae2af
Oct 30 22:08:42.514098 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:08:42.514058    6133 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-2e5a75ca98/isvc-canary-rollback-predictor-75dd485c97-g9nb8" event={"ID":"6257bde3-8c42-42ab-ab0c-b74521d57bb0","Type":"ContainerStarted","Data":"2bee91fe769ddf638bb7bb97fbfb8224858f5d0a41f684d94fff4a1a6116f06d"}
Oct 30 22:08:42.514490 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:08:42.514121    6133 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-2e5a75ca98/isvc-canary-rollback-predictor-75dd485c97-g9nb8" event={"ID":"6257bde3-8c42-42ab-ab0c-b74521d57bb0","Type":"ContainerStarted","Data":"6d08538cb2b4e5e5ca80a9e47ac3259718cdb25b537d5c2708eeddb7e76ae2af"}
Oct 30 22:08:43.141620 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:08:43.141566    6133 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-2e5a75ca98/isvc-canary-rollback-v2-predictor-5f45d9c89b-tgdq4" podUID="db99ab82-2dd5-4af2-9a65-ed086e65ff00" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.18:8643/healthz\": dial tcp 10.134.0.18:8643: connect: connection refused"
Oct 30 22:08:43.145896 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:08:43.145856    6133 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-2e5a75ca98/isvc-canary-rollback-v2-predictor-5f45d9c89b-tgdq4" podUID="db99ab82-2dd5-4af2-9a65-ed086e65ff00" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.18:8080: connect: connection refused"
Oct 30 22:08:46.363514 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:08:46.363481    6133 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-2e5a75ca98/isvc-canary-rollback-v2-predictor-5f45d9c89b-tgdq4"
Oct 30 22:08:46.492015 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:08:46.491902    6133 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/db99ab82-2dd5-4af2-9a65-ed086e65ff00-kube-api-access-j2cnw\" (UniqueName: \"kubernetes.io/projected/db99ab82-2dd5-4af2-9a65-ed086e65ff00-kube-api-access-j2cnw\") pod \"db99ab82-2dd5-4af2-9a65-ed086e65ff00\" (UID: \"db99ab82-2dd5-4af2-9a65-ed086e65ff00\") "
Oct 30 22:08:46.492015 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:08:46.491942    6133 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/db99ab82-2dd5-4af2-9a65-ed086e65ff00-cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/db99ab82-2dd5-4af2-9a65-ed086e65ff00-cabundle-cert\") pod \"db99ab82-2dd5-4af2-9a65-ed086e65ff00\" (UID: \"db99ab82-2dd5-4af2-9a65-ed086e65ff00\") "
Oct 30 22:08:46.492257 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:08:46.492018    6133 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/db99ab82-2dd5-4af2-9a65-ed086e65ff00-kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/db99ab82-2dd5-4af2-9a65-ed086e65ff00-kube-rbac-proxy-sar-config\") pod \"db99ab82-2dd5-4af2-9a65-ed086e65ff00\" (UID: \"db99ab82-2dd5-4af2-9a65-ed086e65ff00\") "
Oct 30 22:08:46.492257 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:08:46.492070    6133 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/db99ab82-2dd5-4af2-9a65-ed086e65ff00-proxy-tls\" (UniqueName: \"kubernetes.io/secret/db99ab82-2dd5-4af2-9a65-ed086e65ff00-proxy-tls\") pod \"db99ab82-2dd5-4af2-9a65-ed086e65ff00\" (UID: \"db99ab82-2dd5-4af2-9a65-ed086e65ff00\") "
Oct 30 22:08:46.492257 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:08:46.492159    6133 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/db99ab82-2dd5-4af2-9a65-ed086e65ff00-kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/db99ab82-2dd5-4af2-9a65-ed086e65ff00-kserve-provision-location\") pod \"db99ab82-2dd5-4af2-9a65-ed086e65ff00\" (UID: \"db99ab82-2dd5-4af2-9a65-ed086e65ff00\") "
Oct 30 22:08:46.492409 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:08:46.492326    6133 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/db99ab82-2dd5-4af2-9a65-ed086e65ff00-cabundle-cert" pod "db99ab82-2dd5-4af2-9a65-ed086e65ff00" (UID: "db99ab82-2dd5-4af2-9a65-ed086e65ff00"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:08:46.492462 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:08:46.492424    6133 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/db99ab82-2dd5-4af2-9a65-ed086e65ff00-cabundle-cert\") on node \"ip-10-0-143-246.ec2.internal\" DevicePath \"\""
Oct 30 22:08:46.492653 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:08:46.492604    6133 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/db99ab82-2dd5-4af2-9a65-ed086e65ff00-kserve-provision-location" pod "db99ab82-2dd5-4af2-9a65-ed086e65ff00" (UID: "db99ab82-2dd5-4af2-9a65-ed086e65ff00"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:08:46.492754 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:08:46.492686    6133 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/db99ab82-2dd5-4af2-9a65-ed086e65ff00-kube-rbac-proxy-sar-config" pod "db99ab82-2dd5-4af2-9a65-ed086e65ff00" (UID: "db99ab82-2dd5-4af2-9a65-ed086e65ff00"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:08:46.495562 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:08:46.495533    6133 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/db99ab82-2dd5-4af2-9a65-ed086e65ff00-kube-api-access-j2cnw" pod "db99ab82-2dd5-4af2-9a65-ed086e65ff00" (UID: "db99ab82-2dd5-4af2-9a65-ed086e65ff00"). InnerVolumeSpecName "kube-api-access-j2cnw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:08:46.495684 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:08:46.495610    6133 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/db99ab82-2dd5-4af2-9a65-ed086e65ff00-proxy-tls" pod "db99ab82-2dd5-4af2-9a65-ed086e65ff00" (UID: "db99ab82-2dd5-4af2-9a65-ed086e65ff00"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:08:46.523032 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:08:46.522982    6133 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw0-7b6759dd5d/isvc-raw-sklearn-batcher-f9cfa-predictor-57b5c6b7cb-ztjg5"]
Oct 30 22:08:46.523317 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:08:46.523291    6133 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw0-7b6759dd5d/isvc-raw-sklearn-batcher-f9cfa-predictor-57b5c6b7cb-ztjg5" podUID="de778448-704c-4d67-84f4-c66dfb1abb14" containerName="kserve-container" containerID="cri-o://1fd68968c7c8bc62402be1ef466c06389e3aec6ef8c140d2f52fdaa9a5a2a828" gracePeriod=30
Oct 30 22:08:46.523408 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:08:46.523364    6133 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw0-7b6759dd5d/isvc-raw-sklearn-batcher-f9cfa-predictor-57b5c6b7cb-ztjg5" podUID="de778448-704c-4d67-84f4-c66dfb1abb14" containerName="agent" containerID="cri-o://215b67b68b881759f2ad3eedf42e532096c674e0e22d99bec8d6fd8b968be788" gracePeriod=30
Oct 30 22:08:46.523502 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:08:46.523409    6133 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw0-7b6759dd5d/isvc-raw-sklearn-batcher-f9cfa-predictor-57b5c6b7cb-ztjg5" podUID="de778448-704c-4d67-84f4-c66dfb1abb14" containerName="kube-rbac-proxy" containerID="cri-o://0a60bf4fb900b85fd05b56caf63f539b4ad305ffc1c9a1b550ef39c0bc85ea05" gracePeriod=30
Oct 30 22:08:46.532224 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:08:46.532190    6133 generic.go:356] "Generic (PLEG): container finished" podID="db99ab82-2dd5-4af2-9a65-ed086e65ff00" containerID="3aa09d0aa80d1567295db8eb67a1478e0dff6e56fa6e0b033a62d0c213003085" exitCode=0
Oct 30 22:08:46.532383 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:08:46.532253    6133 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-2e5a75ca98/isvc-canary-rollback-v2-predictor-5f45d9c89b-tgdq4"
Oct 30 22:08:46.532383 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:08:46.532278    6133 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-2e5a75ca98/isvc-canary-rollback-v2-predictor-5f45d9c89b-tgdq4" event={"ID":"db99ab82-2dd5-4af2-9a65-ed086e65ff00","Type":"ContainerDied","Data":"3aa09d0aa80d1567295db8eb67a1478e0dff6e56fa6e0b033a62d0c213003085"}
Oct 30 22:08:46.532383 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:08:46.532345    6133 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-2e5a75ca98/isvc-canary-rollback-v2-predictor-5f45d9c89b-tgdq4" event={"ID":"db99ab82-2dd5-4af2-9a65-ed086e65ff00","Type":"ContainerDied","Data":"4744899b425d0a7a521350474bec051add2955bd4758fbee8f6f5d752d77ff10"}
Oct 30 22:08:46.532383 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:08:46.532380    6133 scope.go:122] "RemoveContainer" containerID="ac6d167373566d1cae1456a1f34847d197fd84c9b09d142e7d6b35ec5c3c5db4"
Oct 30 22:08:46.534096 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:08:46.534073    6133 generic.go:356] "Generic (PLEG): container finished" podID="6257bde3-8c42-42ab-ab0c-b74521d57bb0" containerID="2bee91fe769ddf638bb7bb97fbfb8224858f5d0a41f684d94fff4a1a6116f06d" exitCode=0
Oct 30 22:08:46.534227 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:08:46.534116    6133 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-2e5a75ca98/isvc-canary-rollback-predictor-75dd485c97-g9nb8" event={"ID":"6257bde3-8c42-42ab-ab0c-b74521d57bb0","Type":"ContainerDied","Data":"2bee91fe769ddf638bb7bb97fbfb8224858f5d0a41f684d94fff4a1a6116f06d"}
Oct 30 22:08:46.543836 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:08:46.543819    6133 scope.go:122] "RemoveContainer" containerID="3aa09d0aa80d1567295db8eb67a1478e0dff6e56fa6e0b033a62d0c213003085"
Oct 30 22:08:46.593210 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:08:46.593165    6133 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j2cnw\" (UniqueName: \"kubernetes.io/projected/db99ab82-2dd5-4af2-9a65-ed086e65ff00-kube-api-access-j2cnw\") on node \"ip-10-0-143-246.ec2.internal\" DevicePath \"\""
Oct 30 22:08:46.593210 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:08:46.593205    6133 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/db99ab82-2dd5-4af2-9a65-ed086e65ff00-kube-rbac-proxy-sar-config\") on node \"ip-10-0-143-246.ec2.internal\" DevicePath \"\""
Oct 30 22:08:46.593210 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:08:46.593216    6133 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/db99ab82-2dd5-4af2-9a65-ed086e65ff00-proxy-tls\") on node \"ip-10-0-143-246.ec2.internal\" DevicePath \"\""
Oct 30 22:08:46.593210 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:08:46.593225    6133 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/db99ab82-2dd5-4af2-9a65-ed086e65ff00-kserve-provision-location\") on node \"ip-10-0-143-246.ec2.internal\" DevicePath \"\""
Oct 30 22:08:46.606314 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:08:46.606285    6133 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw1-2e5a75ca98/isvc-canary-rollback-v2-predictor-5f45d9c89b-tgdq4"]
Oct 30 22:08:46.611623 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:08:46.611596    6133 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-gw1-2e5a75ca98/isvc-canary-rollback-v2-predictor-5f45d9c89b-tgdq4"]
Oct 30 22:08:46.635768 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:08:46.635746    6133 scope.go:122] "RemoveContainer" containerID="0d1b575a7eea5d35dad92698db35bf1d1f1d92394d80c07eddcc7a095ec09b91"
Oct 30 22:08:46.646926 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:08:46.646903    6133 scope.go:122] "RemoveContainer" containerID="ac6d167373566d1cae1456a1f34847d197fd84c9b09d142e7d6b35ec5c3c5db4"
Oct 30 22:08:46.647377 ip-10-0-143-246 kubenswrapper[6133]: E1030 22:08:46.647352    6133 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ac6d167373566d1cae1456a1f34847d197fd84c9b09d142e7d6b35ec5c3c5db4\": container with ID starting with ac6d167373566d1cae1456a1f34847d197fd84c9b09d142e7d6b35ec5c3c5db4 not found: ID does not exist" containerID="ac6d167373566d1cae1456a1f34847d197fd84c9b09d142e7d6b35ec5c3c5db4"
Oct 30 22:08:46.647481 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:08:46.647387    6133 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ac6d167373566d1cae1456a1f34847d197fd84c9b09d142e7d6b35ec5c3c5db4"} err="failed to get container status \"ac6d167373566d1cae1456a1f34847d197fd84c9b09d142e7d6b35ec5c3c5db4\": rpc error: code = NotFound desc = could not find container \"ac6d167373566d1cae1456a1f34847d197fd84c9b09d142e7d6b35ec5c3c5db4\": container with ID starting with ac6d167373566d1cae1456a1f34847d197fd84c9b09d142e7d6b35ec5c3c5db4 not found: ID does not exist"
Oct 30 22:08:46.647481 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:08:46.647413    6133 scope.go:122] "RemoveContainer" containerID="3aa09d0aa80d1567295db8eb67a1478e0dff6e56fa6e0b033a62d0c213003085"
Oct 30 22:08:46.647719 ip-10-0-143-246 kubenswrapper[6133]: E1030 22:08:46.647698    6133 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3aa09d0aa80d1567295db8eb67a1478e0dff6e56fa6e0b033a62d0c213003085\": container with ID starting with 3aa09d0aa80d1567295db8eb67a1478e0dff6e56fa6e0b033a62d0c213003085 not found: ID does not exist" containerID="3aa09d0aa80d1567295db8eb67a1478e0dff6e56fa6e0b033a62d0c213003085"
Oct 30 22:08:46.647794 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:08:46.647725    6133 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3aa09d0aa80d1567295db8eb67a1478e0dff6e56fa6e0b033a62d0c213003085"} err="failed to get container status \"3aa09d0aa80d1567295db8eb67a1478e0dff6e56fa6e0b033a62d0c213003085\": rpc error: code = NotFound desc = could not find container \"3aa09d0aa80d1567295db8eb67a1478e0dff6e56fa6e0b033a62d0c213003085\": container with ID starting with 3aa09d0aa80d1567295db8eb67a1478e0dff6e56fa6e0b033a62d0c213003085 not found: ID does not exist"
Oct 30 22:08:46.647794 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:08:46.647742    6133 scope.go:122] "RemoveContainer" containerID="0d1b575a7eea5d35dad92698db35bf1d1f1d92394d80c07eddcc7a095ec09b91"
Oct 30 22:08:46.648009 ip-10-0-143-246 kubenswrapper[6133]: E1030 22:08:46.647968    6133 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0d1b575a7eea5d35dad92698db35bf1d1f1d92394d80c07eddcc7a095ec09b91\": container with ID starting with 0d1b575a7eea5d35dad92698db35bf1d1f1d92394d80c07eddcc7a095ec09b91 not found: ID does not exist" containerID="0d1b575a7eea5d35dad92698db35bf1d1f1d92394d80c07eddcc7a095ec09b91"
Oct 30 22:08:46.648088 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:08:46.648011    6133 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0d1b575a7eea5d35dad92698db35bf1d1f1d92394d80c07eddcc7a095ec09b91"} err="failed to get container status \"0d1b575a7eea5d35dad92698db35bf1d1f1d92394d80c07eddcc7a095ec09b91\": rpc error: code = NotFound desc = could not find container \"0d1b575a7eea5d35dad92698db35bf1d1f1d92394d80c07eddcc7a095ec09b91\": container with ID starting with 0d1b575a7eea5d35dad92698db35bf1d1f1d92394d80c07eddcc7a095ec09b91 not found: ID does not exist"
Oct 30 22:08:47.159935 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:08:47.159895    6133 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7b6759dd5d/isvc-raw-sklearn-batcher-f9cfa-predictor-57b5c6b7cb-ztjg5" podUID="de778448-704c-4d67-84f4-c66dfb1abb14" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.17:8643/healthz\": dial tcp 10.134.0.17:8643: connect: connection refused"
Oct 30 22:08:47.351640 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:08:47.351594    6133 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7b6759dd5d/isvc-raw-sklearn-batcher-f9cfa-predictor-57b5c6b7cb-ztjg5" podUID="de778448-704c-4d67-84f4-c66dfb1abb14" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.17:8080: connect: connection refused"
Oct 30 22:08:47.351973 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:08:47.351942    6133 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7b6759dd5d/isvc-raw-sklearn-batcher-f9cfa-predictor-57b5c6b7cb-ztjg5" podUID="de778448-704c-4d67-84f4-c66dfb1abb14" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 22:08:47.355458 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:08:47.355431    6133 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="db99ab82-2dd5-4af2-9a65-ed086e65ff00" path="/var/lib/kubelet/pods/db99ab82-2dd5-4af2-9a65-ed086e65ff00/volumes"
Oct 30 22:08:47.540902 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:08:47.540866    6133 generic.go:356] "Generic (PLEG): container finished" podID="de778448-704c-4d67-84f4-c66dfb1abb14" containerID="0a60bf4fb900b85fd05b56caf63f539b4ad305ffc1c9a1b550ef39c0bc85ea05" exitCode=0
Oct 30 22:08:47.541396 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:08:47.540937    6133 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-7b6759dd5d/isvc-raw-sklearn-batcher-f9cfa-predictor-57b5c6b7cb-ztjg5" event={"ID":"de778448-704c-4d67-84f4-c66dfb1abb14","Type":"ContainerDied","Data":"0a60bf4fb900b85fd05b56caf63f539b4ad305ffc1c9a1b550ef39c0bc85ea05"}
Oct 30 22:08:47.543954 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:08:47.543931    6133 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-2e5a75ca98/isvc-canary-rollback-predictor-75dd485c97-g9nb8" event={"ID":"6257bde3-8c42-42ab-ab0c-b74521d57bb0","Type":"ContainerStarted","Data":"16a816cf2099a4c984680a958299516e5cd554f081253f5342f3bf8aa3a81b15"}
Oct 30 22:08:47.544110 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:08:47.543977    6133 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-2e5a75ca98/isvc-canary-rollback-predictor-75dd485c97-g9nb8" event={"ID":"6257bde3-8c42-42ab-ab0c-b74521d57bb0","Type":"ContainerStarted","Data":"f9b3da46b20cf36de98ccd237ec7d7da8c9881a0e645275b0b049d7c70f388b2"}
Oct 30 22:08:47.544212 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:08:47.544195    6133 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw1-2e5a75ca98/isvc-canary-rollback-predictor-75dd485c97-g9nb8"
Oct 30 22:08:47.589368 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:08:47.589305    6133 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-gw1-2e5a75ca98/isvc-canary-rollback-predictor-75dd485c97-g9nb8" podStartSLOduration=3.717159536 podStartE2EDuration="7.589289645s" totalImagesPullingTime="0s" totalInitContainerRuntime="3.872130109s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:08:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:08:47.588126484 +0000 UTC m=+834.348882388" watchObservedRunningTime="2026-10-30 22:08:47.589289645 +0000 UTC m=+834.350045536"
Oct 30 22:08:48.548300 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:08:48.548269    6133 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw1-2e5a75ca98/isvc-canary-rollback-predictor-75dd485c97-g9nb8"
Oct 30 22:08:48.549468 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:08:48.549443    6133 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-2e5a75ca98/isvc-canary-rollback-predictor-75dd485c97-g9nb8" podUID="6257bde3-8c42-42ab-ab0c-b74521d57bb0" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.20:8080: connect: connection refused"
Oct 30 22:08:49.552042 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:08:49.552005    6133 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-2e5a75ca98/isvc-canary-rollback-predictor-75dd485c97-g9nb8" podUID="6257bde3-8c42-42ab-ab0c-b74521d57bb0" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.20:8080: connect: connection refused"
Oct 30 22:08:52.159905 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:08:52.159865    6133 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7b6759dd5d/isvc-raw-sklearn-batcher-f9cfa-predictor-57b5c6b7cb-ztjg5" podUID="de778448-704c-4d67-84f4-c66dfb1abb14" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.17:8643/healthz\": dial tcp 10.134.0.17:8643: connect: connection refused"
Oct 30 22:08:52.565815 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:08:52.565781    6133 generic.go:356] "Generic (PLEG): container finished" podID="de778448-704c-4d67-84f4-c66dfb1abb14" containerID="1fd68968c7c8bc62402be1ef466c06389e3aec6ef8c140d2f52fdaa9a5a2a828" exitCode=0
Oct 30 22:08:52.566010 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:08:52.565856    6133 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-7b6759dd5d/isvc-raw-sklearn-batcher-f9cfa-predictor-57b5c6b7cb-ztjg5" event={"ID":"de778448-704c-4d67-84f4-c66dfb1abb14","Type":"ContainerDied","Data":"1fd68968c7c8bc62402be1ef466c06389e3aec6ef8c140d2f52fdaa9a5a2a828"}
Oct 30 22:08:54.556212 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:08:54.556185    6133 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw1-2e5a75ca98/isvc-canary-rollback-predictor-75dd485c97-g9nb8"
Oct 30 22:08:54.556682 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:08:54.556658    6133 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-2e5a75ca98/isvc-canary-rollback-predictor-75dd485c97-g9nb8" podUID="6257bde3-8c42-42ab-ab0c-b74521d57bb0" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.20:8080: connect: connection refused"
Oct 30 22:08:57.159379 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:08:57.159338    6133 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7b6759dd5d/isvc-raw-sklearn-batcher-f9cfa-predictor-57b5c6b7cb-ztjg5" podUID="de778448-704c-4d67-84f4-c66dfb1abb14" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.17:8643/healthz\": dial tcp 10.134.0.17:8643: connect: connection refused"
Oct 30 22:08:57.159797 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:08:57.159494    6133 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw0-7b6759dd5d/isvc-raw-sklearn-batcher-f9cfa-predictor-57b5c6b7cb-ztjg5"
Oct 30 22:08:57.351106 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:08:57.351054    6133 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7b6759dd5d/isvc-raw-sklearn-batcher-f9cfa-predictor-57b5c6b7cb-ztjg5" podUID="de778448-704c-4d67-84f4-c66dfb1abb14" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.17:8080: connect: connection refused"
Oct 30 22:08:57.351460 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:08:57.351437    6133 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7b6759dd5d/isvc-raw-sklearn-batcher-f9cfa-predictor-57b5c6b7cb-ztjg5" podUID="de778448-704c-4d67-84f4-c66dfb1abb14" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 22:09:02.159925 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:09:02.159870    6133 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7b6759dd5d/isvc-raw-sklearn-batcher-f9cfa-predictor-57b5c6b7cb-ztjg5" podUID="de778448-704c-4d67-84f4-c66dfb1abb14" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.17:8643/healthz\": dial tcp 10.134.0.17:8643: connect: connection refused"
Oct 30 22:09:04.557525 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:09:04.557481    6133 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-2e5a75ca98/isvc-canary-rollback-predictor-75dd485c97-g9nb8" podUID="6257bde3-8c42-42ab-ab0c-b74521d57bb0" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.20:8080: connect: connection refused"
Oct 30 22:09:07.159659 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:09:07.159616    6133 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7b6759dd5d/isvc-raw-sklearn-batcher-f9cfa-predictor-57b5c6b7cb-ztjg5" podUID="de778448-704c-4d67-84f4-c66dfb1abb14" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.17:8643/healthz\": dial tcp 10.134.0.17:8643: connect: connection refused"
Oct 30 22:09:07.351195 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:09:07.351150    6133 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7b6759dd5d/isvc-raw-sklearn-batcher-f9cfa-predictor-57b5c6b7cb-ztjg5" podUID="de778448-704c-4d67-84f4-c66dfb1abb14" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.17:8080: connect: connection refused"
Oct 30 22:09:07.351497 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:09:07.351471    6133 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7b6759dd5d/isvc-raw-sklearn-batcher-f9cfa-predictor-57b5c6b7cb-ztjg5" podUID="de778448-704c-4d67-84f4-c66dfb1abb14" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 22:09:07.354489 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:09:07.354468    6133 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw0-7b6759dd5d/isvc-raw-sklearn-batcher-f9cfa-predictor-57b5c6b7cb-ztjg5"
Oct 30 22:09:07.354587 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:09:07.354558    6133 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw0-7b6759dd5d/isvc-raw-sklearn-batcher-f9cfa-predictor-57b5c6b7cb-ztjg5"
Oct 30 22:09:09.527222 ip-10-0-143-246 kubenswrapper[6133]: E1030 22:09:09.527182    6133 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6jppj" containerName="extractor"
Oct 30 22:09:12.159206 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:09:12.159158    6133 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7b6759dd5d/isvc-raw-sklearn-batcher-f9cfa-predictor-57b5c6b7cb-ztjg5" podUID="de778448-704c-4d67-84f4-c66dfb1abb14" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.17:8643/healthz\": dial tcp 10.134.0.17:8643: connect: connection refused"
Oct 30 22:09:14.557415 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:09:14.557370    6133 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-2e5a75ca98/isvc-canary-rollback-predictor-75dd485c97-g9nb8" podUID="6257bde3-8c42-42ab-ab0c-b74521d57bb0" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.20:8080: connect: connection refused"
Oct 30 22:09:16.657793 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:09:16.657756    6133 generic.go:356] "Generic (PLEG): container finished" podID="de778448-704c-4d67-84f4-c66dfb1abb14" containerID="215b67b68b881759f2ad3eedf42e532096c674e0e22d99bec8d6fd8b968be788" exitCode=0
Oct 30 22:09:16.658335 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:09:16.657830    6133 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-7b6759dd5d/isvc-raw-sklearn-batcher-f9cfa-predictor-57b5c6b7cb-ztjg5" event={"ID":"de778448-704c-4d67-84f4-c66dfb1abb14","Type":"ContainerDied","Data":"215b67b68b881759f2ad3eedf42e532096c674e0e22d99bec8d6fd8b968be788"}
Oct 30 22:09:16.658335 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:09:16.657919    6133 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-7b6759dd5d/isvc-raw-sklearn-batcher-f9cfa-predictor-57b5c6b7cb-ztjg5" event={"ID":"de778448-704c-4d67-84f4-c66dfb1abb14","Type":"ContainerDied","Data":"39723f73f253b027143ef92706b51c79077827185122cf13911063ce5de7313f"}
Oct 30 22:09:16.658335 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:09:16.657946    6133 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="39723f73f253b027143ef92706b51c79077827185122cf13911063ce5de7313f"
Oct 30 22:09:16.666022 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:09:16.665999    6133 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-7b6759dd5d/isvc-raw-sklearn-batcher-f9cfa-predictor-57b5c6b7cb-ztjg5"
Oct 30 22:09:16.734445 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:09:16.734411    6133 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/de778448-704c-4d67-84f4-c66dfb1abb14-kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/de778448-704c-4d67-84f4-c66dfb1abb14-kserve-provision-location\") pod \"de778448-704c-4d67-84f4-c66dfb1abb14\" (UID: \"de778448-704c-4d67-84f4-c66dfb1abb14\") "
Oct 30 22:09:16.734632 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:09:16.734467    6133 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/de778448-704c-4d67-84f4-c66dfb1abb14-cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/de778448-704c-4d67-84f4-c66dfb1abb14-cabundle-cert\") pod \"de778448-704c-4d67-84f4-c66dfb1abb14\" (UID: \"de778448-704c-4d67-84f4-c66dfb1abb14\") "
Oct 30 22:09:16.734632 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:09:16.734517    6133 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/de778448-704c-4d67-84f4-c66dfb1abb14-kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/de778448-704c-4d67-84f4-c66dfb1abb14-kube-rbac-proxy-sar-config\") pod \"de778448-704c-4d67-84f4-c66dfb1abb14\" (UID: \"de778448-704c-4d67-84f4-c66dfb1abb14\") "
Oct 30 22:09:16.734632 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:09:16.734534    6133 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/de778448-704c-4d67-84f4-c66dfb1abb14-proxy-tls\" (UniqueName: \"kubernetes.io/secret/de778448-704c-4d67-84f4-c66dfb1abb14-proxy-tls\") pod \"de778448-704c-4d67-84f4-c66dfb1abb14\" (UID: \"de778448-704c-4d67-84f4-c66dfb1abb14\") "
Oct 30 22:09:16.734759 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:09:16.734644    6133 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/de778448-704c-4d67-84f4-c66dfb1abb14-kube-api-access-5sld6\" (UniqueName: \"kubernetes.io/projected/de778448-704c-4d67-84f4-c66dfb1abb14-kube-api-access-5sld6\") pod \"de778448-704c-4d67-84f4-c66dfb1abb14\" (UID: \"de778448-704c-4d67-84f4-c66dfb1abb14\") "
Oct 30 22:09:16.734759 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:09:16.734745    6133 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/de778448-704c-4d67-84f4-c66dfb1abb14-kserve-provision-location" pod "de778448-704c-4d67-84f4-c66dfb1abb14" (UID: "de778448-704c-4d67-84f4-c66dfb1abb14"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:09:16.734898 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:09:16.734875    6133 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/de778448-704c-4d67-84f4-c66dfb1abb14-cabundle-cert" pod "de778448-704c-4d67-84f4-c66dfb1abb14" (UID: "de778448-704c-4d67-84f4-c66dfb1abb14"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:09:16.734975 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:09:16.734939    6133 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/de778448-704c-4d67-84f4-c66dfb1abb14-kserve-provision-location\") on node \"ip-10-0-143-246.ec2.internal\" DevicePath \"\""
Oct 30 22:09:16.734975 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:09:16.734960    6133 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/de778448-704c-4d67-84f4-c66dfb1abb14-cabundle-cert\") on node \"ip-10-0-143-246.ec2.internal\" DevicePath \"\""
Oct 30 22:09:16.734975 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:09:16.734938    6133 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/de778448-704c-4d67-84f4-c66dfb1abb14-kube-rbac-proxy-sar-config" pod "de778448-704c-4d67-84f4-c66dfb1abb14" (UID: "de778448-704c-4d67-84f4-c66dfb1abb14"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:09:16.736830 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:09:16.736804    6133 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/de778448-704c-4d67-84f4-c66dfb1abb14-kube-api-access-5sld6" pod "de778448-704c-4d67-84f4-c66dfb1abb14" (UID: "de778448-704c-4d67-84f4-c66dfb1abb14"). InnerVolumeSpecName "kube-api-access-5sld6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:09:16.736907 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:09:16.736825    6133 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/de778448-704c-4d67-84f4-c66dfb1abb14-proxy-tls" pod "de778448-704c-4d67-84f4-c66dfb1abb14" (UID: "de778448-704c-4d67-84f4-c66dfb1abb14"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:09:16.835721 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:09:16.835635    6133 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/de778448-704c-4d67-84f4-c66dfb1abb14-kube-rbac-proxy-sar-config\") on node \"ip-10-0-143-246.ec2.internal\" DevicePath \"\""
Oct 30 22:09:16.835721 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:09:16.835665    6133 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/de778448-704c-4d67-84f4-c66dfb1abb14-proxy-tls\") on node \"ip-10-0-143-246.ec2.internal\" DevicePath \"\""
Oct 30 22:09:16.835721 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:09:16.835675    6133 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5sld6\" (UniqueName: \"kubernetes.io/projected/de778448-704c-4d67-84f4-c66dfb1abb14-kube-api-access-5sld6\") on node \"ip-10-0-143-246.ec2.internal\" DevicePath \"\""
Oct 30 22:09:17.661266 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:09:17.661186    6133 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-7b6759dd5d/isvc-raw-sklearn-batcher-f9cfa-predictor-57b5c6b7cb-ztjg5"
Oct 30 22:09:17.728605 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:09:17.728568    6133 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw0-7b6759dd5d/isvc-raw-sklearn-batcher-f9cfa-predictor-57b5c6b7cb-ztjg5"]
Oct 30 22:09:17.741367 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:09:17.741339    6133 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-gw0-7b6759dd5d/isvc-raw-sklearn-batcher-f9cfa-predictor-57b5c6b7cb-ztjg5"]
Oct 30 22:09:18.632018 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:09:18.631972    6133 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-gw0-7b6759dd5d/isvc-sklearn-graph-raw-3d5b0-predictor-65bcf48db8-gdcfk"]
Oct 30 22:09:18.632313 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:09:18.632299    6133 cpu_manager.go:414] "containerMap: removing container" podUID="de778448-704c-4d67-84f4-c66dfb1abb14" containerName="storage-initializer"
Oct 30 22:09:18.632358 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:09:18.632314    6133 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="de778448-704c-4d67-84f4-c66dfb1abb14" containerName="storage-initializer"
Oct 30 22:09:18.632358 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:09:18.632337    6133 cpu_manager.go:414] "containerMap: removing container" podUID="de778448-704c-4d67-84f4-c66dfb1abb14" containerName="agent"
Oct 30 22:09:18.632358 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:09:18.632342    6133 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="de778448-704c-4d67-84f4-c66dfb1abb14" containerName="agent"
Oct 30 22:09:18.632358 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:09:18.632353    6133 cpu_manager.go:414] "containerMap: removing container" podUID="de778448-704c-4d67-84f4-c66dfb1abb14" containerName="kserve-container"
Oct 30 22:09:18.632358 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:09:18.632358    6133 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="de778448-704c-4d67-84f4-c66dfb1abb14" containerName="kserve-container"
Oct 30 22:09:18.632507 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:09:18.632369    6133 cpu_manager.go:414] "containerMap: removing container" podUID="db99ab82-2dd5-4af2-9a65-ed086e65ff00" containerName="storage-initializer"
Oct 30 22:09:18.632507 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:09:18.632374    6133 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="db99ab82-2dd5-4af2-9a65-ed086e65ff00" containerName="storage-initializer"
Oct 30 22:09:18.632507 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:09:18.632381    6133 cpu_manager.go:414] "containerMap: removing container" podUID="db99ab82-2dd5-4af2-9a65-ed086e65ff00" containerName="kube-rbac-proxy"
Oct 30 22:09:18.632507 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:09:18.632386    6133 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="db99ab82-2dd5-4af2-9a65-ed086e65ff00" containerName="kube-rbac-proxy"
Oct 30 22:09:18.632507 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:09:18.632393    6133 cpu_manager.go:414] "containerMap: removing container" podUID="de778448-704c-4d67-84f4-c66dfb1abb14" containerName="kube-rbac-proxy"
Oct 30 22:09:18.632507 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:09:18.632397    6133 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="de778448-704c-4d67-84f4-c66dfb1abb14" containerName="kube-rbac-proxy"
Oct 30 22:09:18.632507 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:09:18.632404    6133 cpu_manager.go:414] "containerMap: removing container" podUID="db99ab82-2dd5-4af2-9a65-ed086e65ff00" containerName="kserve-container"
Oct 30 22:09:18.632507 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:09:18.632408    6133 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="db99ab82-2dd5-4af2-9a65-ed086e65ff00" containerName="kserve-container"
Oct 30 22:09:18.632507 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:09:18.632450    6133 memory_manager.go:365] "RemoveStaleState removing state" podUID="de778448-704c-4d67-84f4-c66dfb1abb14" containerName="kserve-container"
Oct 30 22:09:18.632507 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:09:18.632456    6133 memory_manager.go:365] "RemoveStaleState removing state" podUID="db99ab82-2dd5-4af2-9a65-ed086e65ff00" containerName="kube-rbac-proxy"
Oct 30 22:09:18.632507 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:09:18.632463    6133 memory_manager.go:365] "RemoveStaleState removing state" podUID="de778448-704c-4d67-84f4-c66dfb1abb14" containerName="kube-rbac-proxy"
Oct 30 22:09:18.632507 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:09:18.632468    6133 memory_manager.go:365] "RemoveStaleState removing state" podUID="db99ab82-2dd5-4af2-9a65-ed086e65ff00" containerName="kserve-container"
Oct 30 22:09:18.632507 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:09:18.632476    6133 memory_manager.go:365] "RemoveStaleState removing state" podUID="de778448-704c-4d67-84f4-c66dfb1abb14" containerName="agent"
Oct 30 22:09:18.635685 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:09:18.635664    6133 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-7b6759dd5d/isvc-sklearn-graph-raw-3d5b0-predictor-65bcf48db8-gdcfk"
Oct 30 22:09:18.656245 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:09:18.656217    6133 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw0-7b6759dd5d\"/\"seaweedfs-s3-creds\""
Oct 30 22:09:18.664693 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:09:18.664612    6133 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw0-7b6759dd5d\"/\"kube-root-ca.crt\""
Oct 30 22:09:18.664693 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:09:18.664617    6133 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw0-7b6759dd5d\"/\"isvc-sklearn-graph-raw-3d5b0-predictor-serving-cert\""
Oct 30 22:09:18.664693 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:09:18.664648    6133 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw0-7b6759dd5d\"/\"odh-kserve-custom-ca-bundle\""
Oct 30 22:09:18.664693 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:09:18.664660    6133 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw0-7b6759dd5d\"/\"openshift-service-ca.crt\""
Oct 30 22:09:18.664693 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:09:18.664649    6133 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw0-7b6759dd5d\"/\"isvc-sklearn-graph-raw-3d5b0-kube-rbac-proxy-sar-config\""
Oct 30 22:09:18.664693 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:09:18.664679    6133 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw0-7b6759dd5d\"/\"default-dockercfg-4b7xh\""
Oct 30 22:09:18.669694 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:09:18.669669    6133 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw0-7b6759dd5d/isvc-sklearn-graph-raw-3d5b0-predictor-65bcf48db8-gdcfk"]
Oct 30 22:09:18.753603 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:09:18.753568    6133 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/31df3cd4-f8e0-4dee-821a-919d8702200f-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-graph-raw-3d5b0-predictor-65bcf48db8-gdcfk\" (UID: \"31df3cd4-f8e0-4dee-821a-919d8702200f\") " pod="e2e-gw0-7b6759dd5d/isvc-sklearn-graph-raw-3d5b0-predictor-65bcf48db8-gdcfk"
Oct 30 22:09:18.753603 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:09:18.753608    6133 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/31df3cd4-f8e0-4dee-821a-919d8702200f-kserve-provision-location\") pod \"isvc-sklearn-graph-raw-3d5b0-predictor-65bcf48db8-gdcfk\" (UID: \"31df3cd4-f8e0-4dee-821a-919d8702200f\") " pod="e2e-gw0-7b6759dd5d/isvc-sklearn-graph-raw-3d5b0-predictor-65bcf48db8-gdcfk"
Oct 30 22:09:18.753856 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:09:18.753715    6133 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/31df3cd4-f8e0-4dee-821a-919d8702200f-cabundle-cert\") pod \"isvc-sklearn-graph-raw-3d5b0-predictor-65bcf48db8-gdcfk\" (UID: \"31df3cd4-f8e0-4dee-821a-919d8702200f\") " pod="e2e-gw0-7b6759dd5d/isvc-sklearn-graph-raw-3d5b0-predictor-65bcf48db8-gdcfk"
Oct 30 22:09:18.753856 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:09:18.753764    6133 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/31df3cd4-f8e0-4dee-821a-919d8702200f-proxy-tls\") pod \"isvc-sklearn-graph-raw-3d5b0-predictor-65bcf48db8-gdcfk\" (UID: \"31df3cd4-f8e0-4dee-821a-919d8702200f\") " pod="e2e-gw0-7b6759dd5d/isvc-sklearn-graph-raw-3d5b0-predictor-65bcf48db8-gdcfk"
Oct 30 22:09:18.753856 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:09:18.753833    6133 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jq8wj\" (UniqueName: \"kubernetes.io/projected/31df3cd4-f8e0-4dee-821a-919d8702200f-kube-api-access-jq8wj\") pod \"isvc-sklearn-graph-raw-3d5b0-predictor-65bcf48db8-gdcfk\" (UID: \"31df3cd4-f8e0-4dee-821a-919d8702200f\") " pod="e2e-gw0-7b6759dd5d/isvc-sklearn-graph-raw-3d5b0-predictor-65bcf48db8-gdcfk"
Oct 30 22:09:18.855275 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:09:18.855234    6133 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/31df3cd4-f8e0-4dee-821a-919d8702200f-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-graph-raw-3d5b0-predictor-65bcf48db8-gdcfk\" (UID: \"31df3cd4-f8e0-4dee-821a-919d8702200f\") " pod="e2e-gw0-7b6759dd5d/isvc-sklearn-graph-raw-3d5b0-predictor-65bcf48db8-gdcfk"
Oct 30 22:09:18.855469 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:09:18.855281    6133 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/31df3cd4-f8e0-4dee-821a-919d8702200f-kserve-provision-location\") pod \"isvc-sklearn-graph-raw-3d5b0-predictor-65bcf48db8-gdcfk\" (UID: \"31df3cd4-f8e0-4dee-821a-919d8702200f\") " pod="e2e-gw0-7b6759dd5d/isvc-sklearn-graph-raw-3d5b0-predictor-65bcf48db8-gdcfk"
Oct 30 22:09:18.855469 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:09:18.855337    6133 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/31df3cd4-f8e0-4dee-821a-919d8702200f-cabundle-cert\") pod \"isvc-sklearn-graph-raw-3d5b0-predictor-65bcf48db8-gdcfk\" (UID: \"31df3cd4-f8e0-4dee-821a-919d8702200f\") " pod="e2e-gw0-7b6759dd5d/isvc-sklearn-graph-raw-3d5b0-predictor-65bcf48db8-gdcfk"
Oct 30 22:09:18.855469 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:09:18.855361    6133 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/31df3cd4-f8e0-4dee-821a-919d8702200f-proxy-tls\") pod \"isvc-sklearn-graph-raw-3d5b0-predictor-65bcf48db8-gdcfk\" (UID: \"31df3cd4-f8e0-4dee-821a-919d8702200f\") " pod="e2e-gw0-7b6759dd5d/isvc-sklearn-graph-raw-3d5b0-predictor-65bcf48db8-gdcfk"
Oct 30 22:09:18.855469 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:09:18.855381    6133 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jq8wj\" (UniqueName: \"kubernetes.io/projected/31df3cd4-f8e0-4dee-821a-919d8702200f-kube-api-access-jq8wj\") pod \"isvc-sklearn-graph-raw-3d5b0-predictor-65bcf48db8-gdcfk\" (UID: \"31df3cd4-f8e0-4dee-821a-919d8702200f\") " pod="e2e-gw0-7b6759dd5d/isvc-sklearn-graph-raw-3d5b0-predictor-65bcf48db8-gdcfk"
Oct 30 22:09:18.855819 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:09:18.855794    6133 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/31df3cd4-f8e0-4dee-821a-919d8702200f-kserve-provision-location\") pod \"isvc-sklearn-graph-raw-3d5b0-predictor-65bcf48db8-gdcfk\" (UID: \"31df3cd4-f8e0-4dee-821a-919d8702200f\") " pod="e2e-gw0-7b6759dd5d/isvc-sklearn-graph-raw-3d5b0-predictor-65bcf48db8-gdcfk"
Oct 30 22:09:18.856013 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:09:18.855966    6133 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/31df3cd4-f8e0-4dee-821a-919d8702200f-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-graph-raw-3d5b0-predictor-65bcf48db8-gdcfk\" (UID: \"31df3cd4-f8e0-4dee-821a-919d8702200f\") " pod="e2e-gw0-7b6759dd5d/isvc-sklearn-graph-raw-3d5b0-predictor-65bcf48db8-gdcfk"
Oct 30 22:09:18.856066 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:09:18.856054    6133 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/31df3cd4-f8e0-4dee-821a-919d8702200f-cabundle-cert\") pod \"isvc-sklearn-graph-raw-3d5b0-predictor-65bcf48db8-gdcfk\" (UID: \"31df3cd4-f8e0-4dee-821a-919d8702200f\") " pod="e2e-gw0-7b6759dd5d/isvc-sklearn-graph-raw-3d5b0-predictor-65bcf48db8-gdcfk"
Oct 30 22:09:18.857738 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:09:18.857716    6133 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/31df3cd4-f8e0-4dee-821a-919d8702200f-proxy-tls\") pod \"isvc-sklearn-graph-raw-3d5b0-predictor-65bcf48db8-gdcfk\" (UID: \"31df3cd4-f8e0-4dee-821a-919d8702200f\") " pod="e2e-gw0-7b6759dd5d/isvc-sklearn-graph-raw-3d5b0-predictor-65bcf48db8-gdcfk"
Oct 30 22:09:18.879399 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:09:18.879367    6133 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jq8wj\" (UniqueName: \"kubernetes.io/projected/31df3cd4-f8e0-4dee-821a-919d8702200f-kube-api-access-jq8wj\") pod \"isvc-sklearn-graph-raw-3d5b0-predictor-65bcf48db8-gdcfk\" (UID: \"31df3cd4-f8e0-4dee-821a-919d8702200f\") " pod="e2e-gw0-7b6759dd5d/isvc-sklearn-graph-raw-3d5b0-predictor-65bcf48db8-gdcfk"
Oct 30 22:09:18.952282 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:09:18.952201    6133 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-7b6759dd5d/isvc-sklearn-graph-raw-3d5b0-predictor-65bcf48db8-gdcfk"
Oct 30 22:09:19.029849 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:09:19.029819    6133 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-gw0-7b6759dd5d/isvc-xgboost-graph-raw-3d5b0-predictor-5bc48dbc76-l4lq8"]
Oct 30 22:09:19.034477 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:09:19.034450    6133 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-7b6759dd5d/isvc-xgboost-graph-raw-3d5b0-predictor-5bc48dbc76-l4lq8"
Oct 30 22:09:19.052624 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:09:19.052596    6133 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw0-7b6759dd5d\"/\"isvc-xgboost-graph-raw-3d5b0-predictor-serving-cert\""
Oct 30 22:09:19.052762 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:09:19.052648    6133 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw0-7b6759dd5d\"/\"isvc-xgboost-graph-raw-3d5b0-kube-rbac-proxy-sar-config\""
Oct 30 22:09:19.062427 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:09:19.062398    6133 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw0-7b6759dd5d/isvc-xgboost-graph-raw-3d5b0-predictor-5bc48dbc76-l4lq8"]
Oct 30 22:09:19.157716 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:09:19.157678    6133 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/5e6ed395-7d35-4ec4-aef1-da4c4fe4959a-proxy-tls\") pod \"isvc-xgboost-graph-raw-3d5b0-predictor-5bc48dbc76-l4lq8\" (UID: \"5e6ed395-7d35-4ec4-aef1-da4c4fe4959a\") " pod="e2e-gw0-7b6759dd5d/isvc-xgboost-graph-raw-3d5b0-predictor-5bc48dbc76-l4lq8"
Oct 30 22:09:19.157884 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:09:19.157801    6133 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9jlvr\" (UniqueName: \"kubernetes.io/projected/5e6ed395-7d35-4ec4-aef1-da4c4fe4959a-kube-api-access-9jlvr\") pod \"isvc-xgboost-graph-raw-3d5b0-predictor-5bc48dbc76-l4lq8\" (UID: \"5e6ed395-7d35-4ec4-aef1-da4c4fe4959a\") " pod="e2e-gw0-7b6759dd5d/isvc-xgboost-graph-raw-3d5b0-predictor-5bc48dbc76-l4lq8"
Oct 30 22:09:19.157884 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:09:19.157832    6133 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/5e6ed395-7d35-4ec4-aef1-da4c4fe4959a-cabundle-cert\") pod \"isvc-xgboost-graph-raw-3d5b0-predictor-5bc48dbc76-l4lq8\" (UID: \"5e6ed395-7d35-4ec4-aef1-da4c4fe4959a\") " pod="e2e-gw0-7b6759dd5d/isvc-xgboost-graph-raw-3d5b0-predictor-5bc48dbc76-l4lq8"
Oct 30 22:09:19.157884 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:09:19.157853    6133 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/5e6ed395-7d35-4ec4-aef1-da4c4fe4959a-kserve-provision-location\") pod \"isvc-xgboost-graph-raw-3d5b0-predictor-5bc48dbc76-l4lq8\" (UID: \"5e6ed395-7d35-4ec4-aef1-da4c4fe4959a\") " pod="e2e-gw0-7b6759dd5d/isvc-xgboost-graph-raw-3d5b0-predictor-5bc48dbc76-l4lq8"
Oct 30 22:09:19.157884 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:09:19.157876    6133 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/5e6ed395-7d35-4ec4-aef1-da4c4fe4959a-kube-rbac-proxy-sar-config\") pod \"isvc-xgboost-graph-raw-3d5b0-predictor-5bc48dbc76-l4lq8\" (UID: \"5e6ed395-7d35-4ec4-aef1-da4c4fe4959a\") " pod="e2e-gw0-7b6759dd5d/isvc-xgboost-graph-raw-3d5b0-predictor-5bc48dbc76-l4lq8"
Oct 30 22:09:19.259031 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:09:19.258979    6133 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/5e6ed395-7d35-4ec4-aef1-da4c4fe4959a-proxy-tls\") pod \"isvc-xgboost-graph-raw-3d5b0-predictor-5bc48dbc76-l4lq8\" (UID: \"5e6ed395-7d35-4ec4-aef1-da4c4fe4959a\") " pod="e2e-gw0-7b6759dd5d/isvc-xgboost-graph-raw-3d5b0-predictor-5bc48dbc76-l4lq8"
Oct 30 22:09:19.259031 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:09:19.259018    6133 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw0-7b6759dd5d/isvc-sklearn-graph-raw-3d5b0-predictor-65bcf48db8-gdcfk"]
Oct 30 22:09:19.259200 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:09:19.259079    6133 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9jlvr\" (UniqueName: \"kubernetes.io/projected/5e6ed395-7d35-4ec4-aef1-da4c4fe4959a-kube-api-access-9jlvr\") pod \"isvc-xgboost-graph-raw-3d5b0-predictor-5bc48dbc76-l4lq8\" (UID: \"5e6ed395-7d35-4ec4-aef1-da4c4fe4959a\") " pod="e2e-gw0-7b6759dd5d/isvc-xgboost-graph-raw-3d5b0-predictor-5bc48dbc76-l4lq8"
Oct 30 22:09:19.259200 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:09:19.259107    6133 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/5e6ed395-7d35-4ec4-aef1-da4c4fe4959a-cabundle-cert\") pod \"isvc-xgboost-graph-raw-3d5b0-predictor-5bc48dbc76-l4lq8\" (UID: \"5e6ed395-7d35-4ec4-aef1-da4c4fe4959a\") " pod="e2e-gw0-7b6759dd5d/isvc-xgboost-graph-raw-3d5b0-predictor-5bc48dbc76-l4lq8"
Oct 30 22:09:19.259200 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:09:19.259126    6133 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/5e6ed395-7d35-4ec4-aef1-da4c4fe4959a-kserve-provision-location\") pod \"isvc-xgboost-graph-raw-3d5b0-predictor-5bc48dbc76-l4lq8\" (UID: \"5e6ed395-7d35-4ec4-aef1-da4c4fe4959a\") " pod="e2e-gw0-7b6759dd5d/isvc-xgboost-graph-raw-3d5b0-predictor-5bc48dbc76-l4lq8"
Oct 30 22:09:19.259200 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:09:19.259156    6133 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/5e6ed395-7d35-4ec4-aef1-da4c4fe4959a-kube-rbac-proxy-sar-config\") pod \"isvc-xgboost-graph-raw-3d5b0-predictor-5bc48dbc76-l4lq8\" (UID: \"5e6ed395-7d35-4ec4-aef1-da4c4fe4959a\") " pod="e2e-gw0-7b6759dd5d/isvc-xgboost-graph-raw-3d5b0-predictor-5bc48dbc76-l4lq8"
Oct 30 22:09:19.259592 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:09:19.259547    6133 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/5e6ed395-7d35-4ec4-aef1-da4c4fe4959a-kserve-provision-location\") pod \"isvc-xgboost-graph-raw-3d5b0-predictor-5bc48dbc76-l4lq8\" (UID: \"5e6ed395-7d35-4ec4-aef1-da4c4fe4959a\") " pod="e2e-gw0-7b6759dd5d/isvc-xgboost-graph-raw-3d5b0-predictor-5bc48dbc76-l4lq8"
Oct 30 22:09:19.259948 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:09:19.259922    6133 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/5e6ed395-7d35-4ec4-aef1-da4c4fe4959a-kube-rbac-proxy-sar-config\") pod \"isvc-xgboost-graph-raw-3d5b0-predictor-5bc48dbc76-l4lq8\" (UID: \"5e6ed395-7d35-4ec4-aef1-da4c4fe4959a\") " pod="e2e-gw0-7b6759dd5d/isvc-xgboost-graph-raw-3d5b0-predictor-5bc48dbc76-l4lq8"
Oct 30 22:09:19.260074 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:09:19.259930    6133 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/5e6ed395-7d35-4ec4-aef1-da4c4fe4959a-cabundle-cert\") pod \"isvc-xgboost-graph-raw-3d5b0-predictor-5bc48dbc76-l4lq8\" (UID: \"5e6ed395-7d35-4ec4-aef1-da4c4fe4959a\") " pod="e2e-gw0-7b6759dd5d/isvc-xgboost-graph-raw-3d5b0-predictor-5bc48dbc76-l4lq8"
Oct 30 22:09:19.260379 ip-10-0-143-246 kubenswrapper[6133]: W1030 22:09:19.260354    6133 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod31df3cd4_f8e0_4dee_821a_919d8702200f.slice/crio-51d53d9daa9fa7f2904ca99cdf1480bd3e7007b3edf99d9ce0423a378e09e3fd WatchSource:0}: Error finding container 51d53d9daa9fa7f2904ca99cdf1480bd3e7007b3edf99d9ce0423a378e09e3fd: Status 404 returned error can't find the container with id 51d53d9daa9fa7f2904ca99cdf1480bd3e7007b3edf99d9ce0423a378e09e3fd
Oct 30 22:09:19.261997 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:09:19.261963    6133 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/5e6ed395-7d35-4ec4-aef1-da4c4fe4959a-proxy-tls\") pod \"isvc-xgboost-graph-raw-3d5b0-predictor-5bc48dbc76-l4lq8\" (UID: \"5e6ed395-7d35-4ec4-aef1-da4c4fe4959a\") " pod="e2e-gw0-7b6759dd5d/isvc-xgboost-graph-raw-3d5b0-predictor-5bc48dbc76-l4lq8"
Oct 30 22:09:19.281042 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:09:19.281003    6133 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9jlvr\" (UniqueName: \"kubernetes.io/projected/5e6ed395-7d35-4ec4-aef1-da4c4fe4959a-kube-api-access-9jlvr\") pod \"isvc-xgboost-graph-raw-3d5b0-predictor-5bc48dbc76-l4lq8\" (UID: \"5e6ed395-7d35-4ec4-aef1-da4c4fe4959a\") " pod="e2e-gw0-7b6759dd5d/isvc-xgboost-graph-raw-3d5b0-predictor-5bc48dbc76-l4lq8"
Oct 30 22:09:19.352923 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:09:19.351574    6133 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-7b6759dd5d/isvc-xgboost-graph-raw-3d5b0-predictor-5bc48dbc76-l4lq8"
Oct 30 22:09:19.365946 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:09:19.365906    6133 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="de778448-704c-4d67-84f4-c66dfb1abb14" path="/var/lib/kubelet/pods/de778448-704c-4d67-84f4-c66dfb1abb14/volumes"
Oct 30 22:09:19.668139 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:09:19.668104    6133 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw0-7b6759dd5d/isvc-xgboost-graph-raw-3d5b0-predictor-5bc48dbc76-l4lq8"]
Oct 30 22:09:19.669530 ip-10-0-143-246 kubenswrapper[6133]: W1030 22:09:19.669505    6133 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5e6ed395_7d35_4ec4_aef1_da4c4fe4959a.slice/crio-b8017acd9dea8b5d4efd1537081f71eb3dbcc4e1977c4968c8ae2dcee42e5782 WatchSource:0}: Error finding container b8017acd9dea8b5d4efd1537081f71eb3dbcc4e1977c4968c8ae2dcee42e5782: Status 404 returned error can't find the container with id b8017acd9dea8b5d4efd1537081f71eb3dbcc4e1977c4968c8ae2dcee42e5782
Oct 30 22:09:19.671851 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:09:19.671825    6133 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-7b6759dd5d/isvc-sklearn-graph-raw-3d5b0-predictor-65bcf48db8-gdcfk" event={"ID":"31df3cd4-f8e0-4dee-821a-919d8702200f","Type":"ContainerStarted","Data":"f25c3e1543264ab4a356827484e62f495f33df5f8c06cb822bc2ecb4268892e5"}
Oct 30 22:09:19.672076 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:09:19.672022    6133 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-7b6759dd5d/isvc-sklearn-graph-raw-3d5b0-predictor-65bcf48db8-gdcfk" event={"ID":"31df3cd4-f8e0-4dee-821a-919d8702200f","Type":"ContainerStarted","Data":"51d53d9daa9fa7f2904ca99cdf1480bd3e7007b3edf99d9ce0423a378e09e3fd"}
Oct 30 22:09:20.678151 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:09:20.678108    6133 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-7b6759dd5d/isvc-xgboost-graph-raw-3d5b0-predictor-5bc48dbc76-l4lq8" event={"ID":"5e6ed395-7d35-4ec4-aef1-da4c4fe4959a","Type":"ContainerStarted","Data":"24b996686d83786eea0c07ccfb8e5dcfa136a3fadf75a9e5f7f790362aec788c"}
Oct 30 22:09:20.678656 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:09:20.678170    6133 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-7b6759dd5d/isvc-xgboost-graph-raw-3d5b0-predictor-5bc48dbc76-l4lq8" event={"ID":"5e6ed395-7d35-4ec4-aef1-da4c4fe4959a","Type":"ContainerStarted","Data":"b8017acd9dea8b5d4efd1537081f71eb3dbcc4e1977c4968c8ae2dcee42e5782"}
Oct 30 22:09:23.693090 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:09:23.693052    6133 generic.go:356] "Generic (PLEG): container finished" podID="31df3cd4-f8e0-4dee-821a-919d8702200f" containerID="f25c3e1543264ab4a356827484e62f495f33df5f8c06cb822bc2ecb4268892e5" exitCode=0
Oct 30 22:09:23.693528 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:09:23.693125    6133 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-7b6759dd5d/isvc-sklearn-graph-raw-3d5b0-predictor-65bcf48db8-gdcfk" event={"ID":"31df3cd4-f8e0-4dee-821a-919d8702200f","Type":"ContainerDied","Data":"f25c3e1543264ab4a356827484e62f495f33df5f8c06cb822bc2ecb4268892e5"}
Oct 30 22:09:23.694817 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:09:23.694793    6133 generic.go:356] "Generic (PLEG): container finished" podID="5e6ed395-7d35-4ec4-aef1-da4c4fe4959a" containerID="24b996686d83786eea0c07ccfb8e5dcfa136a3fadf75a9e5f7f790362aec788c" exitCode=0
Oct 30 22:09:23.694910 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:09:23.694842    6133 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-7b6759dd5d/isvc-xgboost-graph-raw-3d5b0-predictor-5bc48dbc76-l4lq8" event={"ID":"5e6ed395-7d35-4ec4-aef1-da4c4fe4959a","Type":"ContainerDied","Data":"24b996686d83786eea0c07ccfb8e5dcfa136a3fadf75a9e5f7f790362aec788c"}
Oct 30 22:09:24.556885 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:09:24.556834    6133 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-2e5a75ca98/isvc-canary-rollback-predictor-75dd485c97-g9nb8" podUID="6257bde3-8c42-42ab-ab0c-b74521d57bb0" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.20:8080: connect: connection refused"
Oct 30 22:09:24.702587 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:09:24.702547    6133 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-7b6759dd5d/isvc-sklearn-graph-raw-3d5b0-predictor-65bcf48db8-gdcfk" event={"ID":"31df3cd4-f8e0-4dee-821a-919d8702200f","Type":"ContainerStarted","Data":"73eac523ca16c17def4506a090fa9f6f1d2eaa98026fc8567850b832176e0295"}
Oct 30 22:09:24.703056 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:09:24.702644    6133 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-7b6759dd5d/isvc-sklearn-graph-raw-3d5b0-predictor-65bcf48db8-gdcfk" event={"ID":"31df3cd4-f8e0-4dee-821a-919d8702200f","Type":"ContainerStarted","Data":"028db74615dc5d0853b4ddd65300c5bdca8572edb5dccc9bf8ad4f5ddac41c3f"}
Oct 30 22:09:24.703782 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:09:24.703759    6133 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw0-7b6759dd5d/isvc-sklearn-graph-raw-3d5b0-predictor-65bcf48db8-gdcfk"
Oct 30 22:09:24.703877 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:09:24.703814    6133 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw0-7b6759dd5d/isvc-sklearn-graph-raw-3d5b0-predictor-65bcf48db8-gdcfk"
Oct 30 22:09:24.705426 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:09:24.705381    6133 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7b6759dd5d/isvc-sklearn-graph-raw-3d5b0-predictor-65bcf48db8-gdcfk" podUID="31df3cd4-f8e0-4dee-821a-919d8702200f" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.21:8080: connect: connection refused"
Oct 30 22:09:24.769113 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:09:24.769044    6133 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-gw0-7b6759dd5d/isvc-sklearn-graph-raw-3d5b0-predictor-65bcf48db8-gdcfk" podStartSLOduration=2.760205361 podStartE2EDuration="6.769024074s" totalImagesPullingTime="0s" totalInitContainerRuntime="4.008818713s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:09:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:09:24.767320537 +0000 UTC m=+871.528076427" watchObservedRunningTime="2026-10-30 22:09:24.769024074 +0000 UTC m=+871.529780023"
Oct 30 22:09:25.708379 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:09:25.708327    6133 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7b6759dd5d/isvc-sklearn-graph-raw-3d5b0-predictor-65bcf48db8-gdcfk" podUID="31df3cd4-f8e0-4dee-821a-919d8702200f" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.21:8080: connect: connection refused"
Oct 30 22:09:30.713411 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:09:30.713378    6133 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw0-7b6759dd5d/isvc-sklearn-graph-raw-3d5b0-predictor-65bcf48db8-gdcfk"
Oct 30 22:09:30.714079 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:09:30.714046    6133 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7b6759dd5d/isvc-sklearn-graph-raw-3d5b0-predictor-65bcf48db8-gdcfk" podUID="31df3cd4-f8e0-4dee-821a-919d8702200f" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.21:8080: connect: connection refused"
Oct 30 22:09:34.557413 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:09:34.557365    6133 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-2e5a75ca98/isvc-canary-rollback-predictor-75dd485c97-g9nb8" podUID="6257bde3-8c42-42ab-ab0c-b74521d57bb0" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.20:8080: connect: connection refused"
Oct 30 22:09:39.535012 ip-10-0-143-246 kubenswrapper[6133]: E1030 22:09:39.534958    6133 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6jppj" containerName="extractor"
Oct 30 22:09:40.714222 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:09:40.714182    6133 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7b6759dd5d/isvc-sklearn-graph-raw-3d5b0-predictor-65bcf48db8-gdcfk" podUID="31df3cd4-f8e0-4dee-821a-919d8702200f" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.21:8080: connect: connection refused"
Oct 30 22:09:43.788926 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:09:43.788891    6133 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-7b6759dd5d/isvc-xgboost-graph-raw-3d5b0-predictor-5bc48dbc76-l4lq8" event={"ID":"5e6ed395-7d35-4ec4-aef1-da4c4fe4959a","Type":"ContainerStarted","Data":"e9a761c8fe0e492c507324bd6c4e75dd1cd86f5b111f5b251bd86889f88c984a"}
Oct 30 22:09:43.789320 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:09:43.789014    6133 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-7b6759dd5d/isvc-xgboost-graph-raw-3d5b0-predictor-5bc48dbc76-l4lq8" event={"ID":"5e6ed395-7d35-4ec4-aef1-da4c4fe4959a","Type":"ContainerStarted","Data":"62b1d91da93fc5b871a14d68c3e19d990b441b1caa1fcb1ad1fb6961bba43df1"}
Oct 30 22:09:43.789320 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:09:43.789264    6133 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw0-7b6759dd5d/isvc-xgboost-graph-raw-3d5b0-predictor-5bc48dbc76-l4lq8"
Oct 30 22:09:43.853342 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:09:43.853290    6133 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-gw0-7b6759dd5d/isvc-xgboost-graph-raw-3d5b0-predictor-5bc48dbc76-l4lq8" podStartSLOduration=3.034375631 podStartE2EDuration="25.853276326s" totalImagesPullingTime="19.039207891s" totalInitContainerRuntime="3.779692804s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:09:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:09:43.852360417 +0000 UTC m=+890.613116330" watchObservedRunningTime="2026-10-30 22:09:43.853276326 +0000 UTC m=+890.614032282"
Oct 30 22:09:44.557012 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:09:44.556964    6133 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-2e5a75ca98/isvc-canary-rollback-predictor-75dd485c97-g9nb8" podUID="6257bde3-8c42-42ab-ab0c-b74521d57bb0" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.20:8080: connect: connection refused"
Oct 30 22:09:44.792591 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:09:44.792557    6133 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw0-7b6759dd5d/isvc-xgboost-graph-raw-3d5b0-predictor-5bc48dbc76-l4lq8"
Oct 30 22:09:44.793782 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:09:44.793755    6133 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7b6759dd5d/isvc-xgboost-graph-raw-3d5b0-predictor-5bc48dbc76-l4lq8" podUID="5e6ed395-7d35-4ec4-aef1-da4c4fe4959a" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.22:8080: connect: connection refused"
Oct 30 22:09:45.796187 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:09:45.796144    6133 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7b6759dd5d/isvc-xgboost-graph-raw-3d5b0-predictor-5bc48dbc76-l4lq8" podUID="5e6ed395-7d35-4ec4-aef1-da4c4fe4959a" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.22:8080: connect: connection refused"
Oct 30 22:09:50.714346 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:09:50.714303    6133 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7b6759dd5d/isvc-sklearn-graph-raw-3d5b0-predictor-65bcf48db8-gdcfk" podUID="31df3cd4-f8e0-4dee-821a-919d8702200f" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.21:8080: connect: connection refused"
Oct 30 22:09:50.800539 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:09:50.800501    6133 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw0-7b6759dd5d/isvc-xgboost-graph-raw-3d5b0-predictor-5bc48dbc76-l4lq8"
Oct 30 22:09:50.801121 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:09:50.801088    6133 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7b6759dd5d/isvc-xgboost-graph-raw-3d5b0-predictor-5bc48dbc76-l4lq8" podUID="5e6ed395-7d35-4ec4-aef1-da4c4fe4959a" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.22:8080: connect: connection refused"
Oct 30 22:09:54.559018 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:09:54.558047    6133 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-2e5a75ca98/isvc-canary-rollback-predictor-75dd485c97-g9nb8" podUID="6257bde3-8c42-42ab-ab0c-b74521d57bb0" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.20:8080: connect: connection refused"
Oct 30 22:10:00.714122 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:10:00.714086    6133 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7b6759dd5d/isvc-sklearn-graph-raw-3d5b0-predictor-65bcf48db8-gdcfk" podUID="31df3cd4-f8e0-4dee-821a-919d8702200f" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.21:8080: connect: connection refused"
Oct 30 22:10:00.801251 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:10:00.801214    6133 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7b6759dd5d/isvc-xgboost-graph-raw-3d5b0-predictor-5bc48dbc76-l4lq8" podUID="5e6ed395-7d35-4ec4-aef1-da4c4fe4959a" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.22:8080: connect: connection refused"
Oct 30 22:10:04.557231 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:10:04.557188    6133 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-2e5a75ca98/isvc-canary-rollback-predictor-75dd485c97-g9nb8" podUID="6257bde3-8c42-42ab-ab0c-b74521d57bb0" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.20:8080: connect: connection refused"
Oct 30 22:10:08.350672 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:10:08.350634    6133 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-2e5a75ca98/isvc-canary-rollback-predictor-75dd485c97-g9nb8" podUID="6257bde3-8c42-42ab-ab0c-b74521d57bb0" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.20:8080: connect: connection refused"
Oct 30 22:10:09.528428 ip-10-0-143-246 kubenswrapper[6133]: E1030 22:10:09.528392    6133 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6jppj" containerName="extractor"
Oct 30 22:10:10.714707 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:10:10.714667    6133 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7b6759dd5d/isvc-sklearn-graph-raw-3d5b0-predictor-65bcf48db8-gdcfk" podUID="31df3cd4-f8e0-4dee-821a-919d8702200f" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.21:8080: connect: connection refused"
Oct 30 22:10:10.801545 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:10:10.801501    6133 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7b6759dd5d/isvc-xgboost-graph-raw-3d5b0-predictor-5bc48dbc76-l4lq8" podUID="5e6ed395-7d35-4ec4-aef1-da4c4fe4959a" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.22:8080: connect: connection refused"
Oct 30 22:10:18.351673 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:10:18.351635    6133 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw1-2e5a75ca98/isvc-canary-rollback-predictor-75dd485c97-g9nb8"
Oct 30 22:10:20.714930 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:10:20.714883    6133 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7b6759dd5d/isvc-sklearn-graph-raw-3d5b0-predictor-65bcf48db8-gdcfk" podUID="31df3cd4-f8e0-4dee-821a-919d8702200f" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.21:8080: connect: connection refused"
Oct 30 22:10:20.801728 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:10:20.801691    6133 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7b6759dd5d/isvc-xgboost-graph-raw-3d5b0-predictor-5bc48dbc76-l4lq8" podUID="5e6ed395-7d35-4ec4-aef1-da4c4fe4959a" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.22:8080: connect: connection refused"
Oct 30 22:10:21.297482 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:10:21.297451    6133 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw1-2e5a75ca98/isvc-canary-rollback-predictor-75dd485c97-g9nb8"]
Oct 30 22:10:21.297712 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:10:21.297688    6133 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw1-2e5a75ca98/isvc-canary-rollback-predictor-75dd485c97-g9nb8" podUID="6257bde3-8c42-42ab-ab0c-b74521d57bb0" containerName="kserve-container" containerID="cri-o://f9b3da46b20cf36de98ccd237ec7d7da8c9881a0e645275b0b049d7c70f388b2" gracePeriod=30
Oct 30 22:10:21.297784 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:10:21.297717    6133 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw1-2e5a75ca98/isvc-canary-rollback-predictor-75dd485c97-g9nb8" podUID="6257bde3-8c42-42ab-ab0c-b74521d57bb0" containerName="kube-rbac-proxy" containerID="cri-o://16a816cf2099a4c984680a958299516e5cd554f081253f5342f3bf8aa3a81b15" gracePeriod=30
Oct 30 22:10:21.941597 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:10:21.941561    6133 generic.go:356] "Generic (PLEG): container finished" podID="6257bde3-8c42-42ab-ab0c-b74521d57bb0" containerID="16a816cf2099a4c984680a958299516e5cd554f081253f5342f3bf8aa3a81b15" exitCode=0
Oct 30 22:10:21.942095 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:10:21.941633    6133 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-2e5a75ca98/isvc-canary-rollback-predictor-75dd485c97-g9nb8" event={"ID":"6257bde3-8c42-42ab-ab0c-b74521d57bb0","Type":"ContainerDied","Data":"16a816cf2099a4c984680a958299516e5cd554f081253f5342f3bf8aa3a81b15"}
Oct 30 22:10:24.552929 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:10:24.552886    6133 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-2e5a75ca98/isvc-canary-rollback-predictor-75dd485c97-g9nb8" podUID="6257bde3-8c42-42ab-ab0c-b74521d57bb0" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.20:8643/healthz\": dial tcp 10.134.0.20:8643: connect: connection refused"
Oct 30 22:10:26.841220 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:10:26.841194    6133 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-2e5a75ca98/isvc-canary-rollback-predictor-75dd485c97-g9nb8"
Oct 30 22:10:26.948929 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:10:26.948841    6133 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/6257bde3-8c42-42ab-ab0c-b74521d57bb0-cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/6257bde3-8c42-42ab-ab0c-b74521d57bb0-cabundle-cert\") pod \"6257bde3-8c42-42ab-ab0c-b74521d57bb0\" (UID: \"6257bde3-8c42-42ab-ab0c-b74521d57bb0\") "
Oct 30 22:10:26.948929 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:10:26.948921    6133 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/6257bde3-8c42-42ab-ab0c-b74521d57bb0-kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/6257bde3-8c42-42ab-ab0c-b74521d57bb0-kube-rbac-proxy-sar-config\") pod \"6257bde3-8c42-42ab-ab0c-b74521d57bb0\" (UID: \"6257bde3-8c42-42ab-ab0c-b74521d57bb0\") "
Oct 30 22:10:26.949152 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:10:26.948957    6133 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/6257bde3-8c42-42ab-ab0c-b74521d57bb0-proxy-tls\" (UniqueName: \"kubernetes.io/secret/6257bde3-8c42-42ab-ab0c-b74521d57bb0-proxy-tls\") pod \"6257bde3-8c42-42ab-ab0c-b74521d57bb0\" (UID: \"6257bde3-8c42-42ab-ab0c-b74521d57bb0\") "
Oct 30 22:10:26.949152 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:10:26.949027    6133 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6257bde3-8c42-42ab-ab0c-b74521d57bb0-kube-api-access-dqz5n\" (UniqueName: \"kubernetes.io/projected/6257bde3-8c42-42ab-ab0c-b74521d57bb0-kube-api-access-dqz5n\") pod \"6257bde3-8c42-42ab-ab0c-b74521d57bb0\" (UID: \"6257bde3-8c42-42ab-ab0c-b74521d57bb0\") "
Oct 30 22:10:26.949152 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:10:26.949059    6133 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6257bde3-8c42-42ab-ab0c-b74521d57bb0-kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/6257bde3-8c42-42ab-ab0c-b74521d57bb0-kserve-provision-location\") pod \"6257bde3-8c42-42ab-ab0c-b74521d57bb0\" (UID: \"6257bde3-8c42-42ab-ab0c-b74521d57bb0\") "
Oct 30 22:10:26.949310 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:10:26.949281    6133 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6257bde3-8c42-42ab-ab0c-b74521d57bb0-cabundle-cert" pod "6257bde3-8c42-42ab-ab0c-b74521d57bb0" (UID: "6257bde3-8c42-42ab-ab0c-b74521d57bb0"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:10:26.949368 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:10:26.949317    6133 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6257bde3-8c42-42ab-ab0c-b74521d57bb0-kube-rbac-proxy-sar-config" pod "6257bde3-8c42-42ab-ab0c-b74521d57bb0" (UID: "6257bde3-8c42-42ab-ab0c-b74521d57bb0"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:10:26.949493 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:10:26.949467    6133 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6257bde3-8c42-42ab-ab0c-b74521d57bb0-kserve-provision-location" pod "6257bde3-8c42-42ab-ab0c-b74521d57bb0" (UID: "6257bde3-8c42-42ab-ab0c-b74521d57bb0"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:10:26.951248 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:10:26.951215    6133 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6257bde3-8c42-42ab-ab0c-b74521d57bb0-kube-api-access-dqz5n" pod "6257bde3-8c42-42ab-ab0c-b74521d57bb0" (UID: "6257bde3-8c42-42ab-ab0c-b74521d57bb0"). InnerVolumeSpecName "kube-api-access-dqz5n". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:10:26.951334 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:10:26.951243    6133 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6257bde3-8c42-42ab-ab0c-b74521d57bb0-proxy-tls" pod "6257bde3-8c42-42ab-ab0c-b74521d57bb0" (UID: "6257bde3-8c42-42ab-ab0c-b74521d57bb0"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:10:26.962707 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:10:26.962679    6133 generic.go:356] "Generic (PLEG): container finished" podID="6257bde3-8c42-42ab-ab0c-b74521d57bb0" containerID="f9b3da46b20cf36de98ccd237ec7d7da8c9881a0e645275b0b049d7c70f388b2" exitCode=0
Oct 30 22:10:26.962849 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:10:26.962714    6133 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-2e5a75ca98/isvc-canary-rollback-predictor-75dd485c97-g9nb8" event={"ID":"6257bde3-8c42-42ab-ab0c-b74521d57bb0","Type":"ContainerDied","Data":"f9b3da46b20cf36de98ccd237ec7d7da8c9881a0e645275b0b049d7c70f388b2"}
Oct 30 22:10:26.962849 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:10:26.962764    6133 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-2e5a75ca98/isvc-canary-rollback-predictor-75dd485c97-g9nb8"
Oct 30 22:10:26.962849 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:10:26.962790    6133 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-2e5a75ca98/isvc-canary-rollback-predictor-75dd485c97-g9nb8" event={"ID":"6257bde3-8c42-42ab-ab0c-b74521d57bb0","Type":"ContainerDied","Data":"6d08538cb2b4e5e5ca80a9e47ac3259718cdb25b537d5c2708eeddb7e76ae2af"}
Oct 30 22:10:26.962849 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:10:26.962818    6133 scope.go:122] "RemoveContainer" containerID="16a816cf2099a4c984680a958299516e5cd554f081253f5342f3bf8aa3a81b15"
Oct 30 22:10:26.971762 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:10:26.971741    6133 scope.go:122] "RemoveContainer" containerID="f9b3da46b20cf36de98ccd237ec7d7da8c9881a0e645275b0b049d7c70f388b2"
Oct 30 22:10:26.980934 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:10:26.980916    6133 scope.go:122] "RemoveContainer" containerID="2bee91fe769ddf638bb7bb97fbfb8224858f5d0a41f684d94fff4a1a6116f06d"
Oct 30 22:10:26.988895 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:10:26.988872    6133 scope.go:122] "RemoveContainer" containerID="16a816cf2099a4c984680a958299516e5cd554f081253f5342f3bf8aa3a81b15"
Oct 30 22:10:26.989231 ip-10-0-143-246 kubenswrapper[6133]: E1030 22:10:26.989211    6133 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"16a816cf2099a4c984680a958299516e5cd554f081253f5342f3bf8aa3a81b15\": container with ID starting with 16a816cf2099a4c984680a958299516e5cd554f081253f5342f3bf8aa3a81b15 not found: ID does not exist" containerID="16a816cf2099a4c984680a958299516e5cd554f081253f5342f3bf8aa3a81b15"
Oct 30 22:10:26.989319 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:10:26.989246    6133 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"16a816cf2099a4c984680a958299516e5cd554f081253f5342f3bf8aa3a81b15"} err="failed to get container status \"16a816cf2099a4c984680a958299516e5cd554f081253f5342f3bf8aa3a81b15\": rpc error: code = NotFound desc = could not find container \"16a816cf2099a4c984680a958299516e5cd554f081253f5342f3bf8aa3a81b15\": container with ID starting with 16a816cf2099a4c984680a958299516e5cd554f081253f5342f3bf8aa3a81b15 not found: ID does not exist"
Oct 30 22:10:26.989319 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:10:26.989272    6133 scope.go:122] "RemoveContainer" containerID="f9b3da46b20cf36de98ccd237ec7d7da8c9881a0e645275b0b049d7c70f388b2"
Oct 30 22:10:26.989557 ip-10-0-143-246 kubenswrapper[6133]: E1030 22:10:26.989540    6133 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f9b3da46b20cf36de98ccd237ec7d7da8c9881a0e645275b0b049d7c70f388b2\": container with ID starting with f9b3da46b20cf36de98ccd237ec7d7da8c9881a0e645275b0b049d7c70f388b2 not found: ID does not exist" containerID="f9b3da46b20cf36de98ccd237ec7d7da8c9881a0e645275b0b049d7c70f388b2"
Oct 30 22:10:26.989600 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:10:26.989564    6133 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f9b3da46b20cf36de98ccd237ec7d7da8c9881a0e645275b0b049d7c70f388b2"} err="failed to get container status \"f9b3da46b20cf36de98ccd237ec7d7da8c9881a0e645275b0b049d7c70f388b2\": rpc error: code = NotFound desc = could not find container \"f9b3da46b20cf36de98ccd237ec7d7da8c9881a0e645275b0b049d7c70f388b2\": container with ID starting with f9b3da46b20cf36de98ccd237ec7d7da8c9881a0e645275b0b049d7c70f388b2 not found: ID does not exist"
Oct 30 22:10:26.989600 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:10:26.989582    6133 scope.go:122] "RemoveContainer" containerID="2bee91fe769ddf638bb7bb97fbfb8224858f5d0a41f684d94fff4a1a6116f06d"
Oct 30 22:10:26.989821 ip-10-0-143-246 kubenswrapper[6133]: E1030 22:10:26.989805    6133 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2bee91fe769ddf638bb7bb97fbfb8224858f5d0a41f684d94fff4a1a6116f06d\": container with ID starting with 2bee91fe769ddf638bb7bb97fbfb8224858f5d0a41f684d94fff4a1a6116f06d not found: ID does not exist" containerID="2bee91fe769ddf638bb7bb97fbfb8224858f5d0a41f684d94fff4a1a6116f06d"
Oct 30 22:10:26.989870 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:10:26.989829    6133 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2bee91fe769ddf638bb7bb97fbfb8224858f5d0a41f684d94fff4a1a6116f06d"} err="failed to get container status \"2bee91fe769ddf638bb7bb97fbfb8224858f5d0a41f684d94fff4a1a6116f06d\": rpc error: code = NotFound desc = could not find container \"2bee91fe769ddf638bb7bb97fbfb8224858f5d0a41f684d94fff4a1a6116f06d\": container with ID starting with 2bee91fe769ddf638bb7bb97fbfb8224858f5d0a41f684d94fff4a1a6116f06d not found: ID does not exist"
Oct 30 22:10:27.018357 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:10:27.018325    6133 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw1-2e5a75ca98/isvc-canary-rollback-predictor-75dd485c97-g9nb8"]
Oct 30 22:10:27.025271 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:10:27.025233    6133 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-gw1-2e5a75ca98/isvc-canary-rollback-predictor-75dd485c97-g9nb8"]
Oct 30 22:10:27.049735 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:10:27.049705    6133 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/6257bde3-8c42-42ab-ab0c-b74521d57bb0-kube-rbac-proxy-sar-config\") on node \"ip-10-0-143-246.ec2.internal\" DevicePath \"\""
Oct 30 22:10:27.049735 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:10:27.049734    6133 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/6257bde3-8c42-42ab-ab0c-b74521d57bb0-proxy-tls\") on node \"ip-10-0-143-246.ec2.internal\" DevicePath \"\""
Oct 30 22:10:27.049933 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:10:27.049747    6133 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dqz5n\" (UniqueName: \"kubernetes.io/projected/6257bde3-8c42-42ab-ab0c-b74521d57bb0-kube-api-access-dqz5n\") on node \"ip-10-0-143-246.ec2.internal\" DevicePath \"\""
Oct 30 22:10:27.049933 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:10:27.049758    6133 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/6257bde3-8c42-42ab-ab0c-b74521d57bb0-kserve-provision-location\") on node \"ip-10-0-143-246.ec2.internal\" DevicePath \"\""
Oct 30 22:10:27.049933 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:10:27.049767    6133 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/6257bde3-8c42-42ab-ab0c-b74521d57bb0-cabundle-cert\") on node \"ip-10-0-143-246.ec2.internal\" DevicePath \"\""
Oct 30 22:10:27.355232 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:10:27.355202    6133 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6257bde3-8c42-42ab-ab0c-b74521d57bb0" path="/var/lib/kubelet/pods/6257bde3-8c42-42ab-ab0c-b74521d57bb0/volumes"
Oct 30 22:10:29.118521 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:10:29.118488    6133 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-gw1-2e5a75ca98/isvc-canary-stop-predictor-7f5d58797c-7dm5f"]
Oct 30 22:10:29.119052 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:10:29.118953    6133 cpu_manager.go:414] "containerMap: removing container" podUID="6257bde3-8c42-42ab-ab0c-b74521d57bb0" containerName="kserve-container"
Oct 30 22:10:29.119052 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:10:29.118971    6133 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6257bde3-8c42-42ab-ab0c-b74521d57bb0" containerName="kserve-container"
Oct 30 22:10:29.119052 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:10:29.119015    6133 cpu_manager.go:414] "containerMap: removing container" podUID="6257bde3-8c42-42ab-ab0c-b74521d57bb0" containerName="storage-initializer"
Oct 30 22:10:29.119052 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:10:29.119025    6133 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6257bde3-8c42-42ab-ab0c-b74521d57bb0" containerName="storage-initializer"
Oct 30 22:10:29.119052 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:10:29.119042    6133 cpu_manager.go:414] "containerMap: removing container" podUID="6257bde3-8c42-42ab-ab0c-b74521d57bb0" containerName="kube-rbac-proxy"
Oct 30 22:10:29.119052 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:10:29.119051    6133 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6257bde3-8c42-42ab-ab0c-b74521d57bb0" containerName="kube-rbac-proxy"
Oct 30 22:10:29.119384 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:10:29.119153    6133 memory_manager.go:365] "RemoveStaleState removing state" podUID="6257bde3-8c42-42ab-ab0c-b74521d57bb0" containerName="kube-rbac-proxy"
Oct 30 22:10:29.119384 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:10:29.119174    6133 memory_manager.go:365] "RemoveStaleState removing state" podUID="6257bde3-8c42-42ab-ab0c-b74521d57bb0" containerName="kserve-container"
Oct 30 22:10:29.122216 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:10:29.122193    6133 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-2e5a75ca98/isvc-canary-stop-predictor-7f5d58797c-7dm5f"
Oct 30 22:10:29.138414 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:10:29.138389    6133 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw1-2e5a75ca98\"/\"isvc-canary-stop-kube-rbac-proxy-sar-config\""
Oct 30 22:10:29.150479 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:10:29.150450    6133 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw1-2e5a75ca98\"/\"kube-root-ca.crt\""
Oct 30 22:10:29.150479 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:10:29.150469    6133 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw1-2e5a75ca98\"/\"seaweedfs-s3-creds\""
Oct 30 22:10:29.150479 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:10:29.150450    6133 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw1-2e5a75ca98\"/\"openshift-service-ca.crt\""
Oct 30 22:10:29.150479 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:10:29.150465    6133 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw1-2e5a75ca98\"/\"odh-kserve-custom-ca-bundle\""
Oct 30 22:10:29.150789 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:10:29.150527    6133 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw1-2e5a75ca98\"/\"isvc-canary-stop-predictor-serving-cert\""
Oct 30 22:10:29.150789 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:10:29.150612    6133 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw1-2e5a75ca98\"/\"default-dockercfg-d9bw2\""
Oct 30 22:10:29.151773 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:10:29.151756    6133 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw1-2e5a75ca98/isvc-canary-stop-predictor-7f5d58797c-7dm5f"]
Oct 30 22:10:29.268375 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:10:29.268342    6133 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/bb80ffe3-13a0-4765-aa4f-4e74eb4f0a42-cabundle-cert\") pod \"isvc-canary-stop-predictor-7f5d58797c-7dm5f\" (UID: \"bb80ffe3-13a0-4765-aa4f-4e74eb4f0a42\") " pod="e2e-gw1-2e5a75ca98/isvc-canary-stop-predictor-7f5d58797c-7dm5f"
Oct 30 22:10:29.268375 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:10:29.268378    6133 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/bb80ffe3-13a0-4765-aa4f-4e74eb4f0a42-kserve-provision-location\") pod \"isvc-canary-stop-predictor-7f5d58797c-7dm5f\" (UID: \"bb80ffe3-13a0-4765-aa4f-4e74eb4f0a42\") " pod="e2e-gw1-2e5a75ca98/isvc-canary-stop-predictor-7f5d58797c-7dm5f"
Oct 30 22:10:29.268606 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:10:29.268460    6133 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6t5h5\" (UniqueName: \"kubernetes.io/projected/bb80ffe3-13a0-4765-aa4f-4e74eb4f0a42-kube-api-access-6t5h5\") pod \"isvc-canary-stop-predictor-7f5d58797c-7dm5f\" (UID: \"bb80ffe3-13a0-4765-aa4f-4e74eb4f0a42\") " pod="e2e-gw1-2e5a75ca98/isvc-canary-stop-predictor-7f5d58797c-7dm5f"
Oct 30 22:10:29.268606 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:10:29.268526    6133 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/bb80ffe3-13a0-4765-aa4f-4e74eb4f0a42-proxy-tls\") pod \"isvc-canary-stop-predictor-7f5d58797c-7dm5f\" (UID: \"bb80ffe3-13a0-4765-aa4f-4e74eb4f0a42\") " pod="e2e-gw1-2e5a75ca98/isvc-canary-stop-predictor-7f5d58797c-7dm5f"
Oct 30 22:10:29.268606 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:10:29.268555    6133 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/bb80ffe3-13a0-4765-aa4f-4e74eb4f0a42-kube-rbac-proxy-sar-config\") pod \"isvc-canary-stop-predictor-7f5d58797c-7dm5f\" (UID: \"bb80ffe3-13a0-4765-aa4f-4e74eb4f0a42\") " pod="e2e-gw1-2e5a75ca98/isvc-canary-stop-predictor-7f5d58797c-7dm5f"
Oct 30 22:10:29.369660 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:10:29.369570    6133 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/bb80ffe3-13a0-4765-aa4f-4e74eb4f0a42-cabundle-cert\") pod \"isvc-canary-stop-predictor-7f5d58797c-7dm5f\" (UID: \"bb80ffe3-13a0-4765-aa4f-4e74eb4f0a42\") " pod="e2e-gw1-2e5a75ca98/isvc-canary-stop-predictor-7f5d58797c-7dm5f"
Oct 30 22:10:29.369660 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:10:29.369614    6133 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/bb80ffe3-13a0-4765-aa4f-4e74eb4f0a42-kserve-provision-location\") pod \"isvc-canary-stop-predictor-7f5d58797c-7dm5f\" (UID: \"bb80ffe3-13a0-4765-aa4f-4e74eb4f0a42\") " pod="e2e-gw1-2e5a75ca98/isvc-canary-stop-predictor-7f5d58797c-7dm5f"
Oct 30 22:10:29.369660 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:10:29.369647    6133 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6t5h5\" (UniqueName: \"kubernetes.io/projected/bb80ffe3-13a0-4765-aa4f-4e74eb4f0a42-kube-api-access-6t5h5\") pod \"isvc-canary-stop-predictor-7f5d58797c-7dm5f\" (UID: \"bb80ffe3-13a0-4765-aa4f-4e74eb4f0a42\") " pod="e2e-gw1-2e5a75ca98/isvc-canary-stop-predictor-7f5d58797c-7dm5f"
Oct 30 22:10:29.369943 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:10:29.369670    6133 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/bb80ffe3-13a0-4765-aa4f-4e74eb4f0a42-proxy-tls\") pod \"isvc-canary-stop-predictor-7f5d58797c-7dm5f\" (UID: \"bb80ffe3-13a0-4765-aa4f-4e74eb4f0a42\") " pod="e2e-gw1-2e5a75ca98/isvc-canary-stop-predictor-7f5d58797c-7dm5f"
Oct 30 22:10:29.369943 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:10:29.369688    6133 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/bb80ffe3-13a0-4765-aa4f-4e74eb4f0a42-kube-rbac-proxy-sar-config\") pod \"isvc-canary-stop-predictor-7f5d58797c-7dm5f\" (UID: \"bb80ffe3-13a0-4765-aa4f-4e74eb4f0a42\") " pod="e2e-gw1-2e5a75ca98/isvc-canary-stop-predictor-7f5d58797c-7dm5f"
Oct 30 22:10:29.370164 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:10:29.370140    6133 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/bb80ffe3-13a0-4765-aa4f-4e74eb4f0a42-kserve-provision-location\") pod \"isvc-canary-stop-predictor-7f5d58797c-7dm5f\" (UID: \"bb80ffe3-13a0-4765-aa4f-4e74eb4f0a42\") " pod="e2e-gw1-2e5a75ca98/isvc-canary-stop-predictor-7f5d58797c-7dm5f"
Oct 30 22:10:29.370396 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:10:29.370371    6133 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/bb80ffe3-13a0-4765-aa4f-4e74eb4f0a42-cabundle-cert\") pod \"isvc-canary-stop-predictor-7f5d58797c-7dm5f\" (UID: \"bb80ffe3-13a0-4765-aa4f-4e74eb4f0a42\") " pod="e2e-gw1-2e5a75ca98/isvc-canary-stop-predictor-7f5d58797c-7dm5f"
Oct 30 22:10:29.370541 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:10:29.370524    6133 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/bb80ffe3-13a0-4765-aa4f-4e74eb4f0a42-kube-rbac-proxy-sar-config\") pod \"isvc-canary-stop-predictor-7f5d58797c-7dm5f\" (UID: \"bb80ffe3-13a0-4765-aa4f-4e74eb4f0a42\") " pod="e2e-gw1-2e5a75ca98/isvc-canary-stop-predictor-7f5d58797c-7dm5f"
Oct 30 22:10:29.372320 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:10:29.372296    6133 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/bb80ffe3-13a0-4765-aa4f-4e74eb4f0a42-proxy-tls\") pod \"isvc-canary-stop-predictor-7f5d58797c-7dm5f\" (UID: \"bb80ffe3-13a0-4765-aa4f-4e74eb4f0a42\") " pod="e2e-gw1-2e5a75ca98/isvc-canary-stop-predictor-7f5d58797c-7dm5f"
Oct 30 22:10:29.391097 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:10:29.391068    6133 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6t5h5\" (UniqueName: \"kubernetes.io/projected/bb80ffe3-13a0-4765-aa4f-4e74eb4f0a42-kube-api-access-6t5h5\") pod \"isvc-canary-stop-predictor-7f5d58797c-7dm5f\" (UID: \"bb80ffe3-13a0-4765-aa4f-4e74eb4f0a42\") " pod="e2e-gw1-2e5a75ca98/isvc-canary-stop-predictor-7f5d58797c-7dm5f"
Oct 30 22:10:29.436104 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:10:29.436058    6133 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-2e5a75ca98/isvc-canary-stop-predictor-7f5d58797c-7dm5f"
Oct 30 22:10:29.741448 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:10:29.741417    6133 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw1-2e5a75ca98/isvc-canary-stop-predictor-7f5d58797c-7dm5f"]
Oct 30 22:10:29.742106 ip-10-0-143-246 kubenswrapper[6133]: W1030 22:10:29.742079    6133 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbb80ffe3_13a0_4765_aa4f_4e74eb4f0a42.slice/crio-af8c2273bf0f73aae26f430681db5883908c40fef58b952583d51b6f301a6a42 WatchSource:0}: Error finding container af8c2273bf0f73aae26f430681db5883908c40fef58b952583d51b6f301a6a42: Status 404 returned error can't find the container with id af8c2273bf0f73aae26f430681db5883908c40fef58b952583d51b6f301a6a42
Oct 30 22:10:29.980287 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:10:29.980244    6133 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-2e5a75ca98/isvc-canary-stop-predictor-7f5d58797c-7dm5f" event={"ID":"bb80ffe3-13a0-4765-aa4f-4e74eb4f0a42","Type":"ContainerStarted","Data":"2faa92a3f45a5d492d9b8fab8c2d6284f8eb997e39b7d8374c016ea57b3736f0"}
Oct 30 22:10:29.980455 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:10:29.980311    6133 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-2e5a75ca98/isvc-canary-stop-predictor-7f5d58797c-7dm5f" event={"ID":"bb80ffe3-13a0-4765-aa4f-4e74eb4f0a42","Type":"ContainerStarted","Data":"af8c2273bf0f73aae26f430681db5883908c40fef58b952583d51b6f301a6a42"}
Oct 30 22:10:30.714035 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:10:30.713975    6133 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7b6759dd5d/isvc-sklearn-graph-raw-3d5b0-predictor-65bcf48db8-gdcfk" podUID="31df3cd4-f8e0-4dee-821a-919d8702200f" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.21:8080: connect: connection refused"
Oct 30 22:10:30.801811 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:10:30.801772    6133 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7b6759dd5d/isvc-xgboost-graph-raw-3d5b0-predictor-5bc48dbc76-l4lq8" podUID="5e6ed395-7d35-4ec4-aef1-da4c4fe4959a" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.22:8080: connect: connection refused"
Oct 30 22:10:33.997117 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:10:33.997081    6133 generic.go:356] "Generic (PLEG): container finished" podID="bb80ffe3-13a0-4765-aa4f-4e74eb4f0a42" containerID="2faa92a3f45a5d492d9b8fab8c2d6284f8eb997e39b7d8374c016ea57b3736f0" exitCode=0
Oct 30 22:10:33.997517 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:10:33.997162    6133 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-2e5a75ca98/isvc-canary-stop-predictor-7f5d58797c-7dm5f" event={"ID":"bb80ffe3-13a0-4765-aa4f-4e74eb4f0a42","Type":"ContainerDied","Data":"2faa92a3f45a5d492d9b8fab8c2d6284f8eb997e39b7d8374c016ea57b3736f0"}
Oct 30 22:10:35.003551 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:10:35.003518    6133 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-2e5a75ca98/isvc-canary-stop-predictor-7f5d58797c-7dm5f" event={"ID":"bb80ffe3-13a0-4765-aa4f-4e74eb4f0a42","Type":"ContainerStarted","Data":"37d57c215a0b1d6a319a925596761d12caebd37558acfc444853ed17cdaee6fc"}
Oct 30 22:10:35.003977 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:10:35.003577    6133 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-2e5a75ca98/isvc-canary-stop-predictor-7f5d58797c-7dm5f" event={"ID":"bb80ffe3-13a0-4765-aa4f-4e74eb4f0a42","Type":"ContainerStarted","Data":"3e5ed522261609c1606663c2c8ebfc918a4ec347764b87644bb40025e89c0998"}
Oct 30 22:10:35.003977 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:10:35.003774    6133 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw1-2e5a75ca98/isvc-canary-stop-predictor-7f5d58797c-7dm5f"
Oct 30 22:10:35.063220 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:10:35.063161    6133 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-gw1-2e5a75ca98/isvc-canary-stop-predictor-7f5d58797c-7dm5f" podStartSLOduration=2.322578905 podStartE2EDuration="6.063146638s" totalImagesPullingTime="0s" totalInitContainerRuntime="3.740567733s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:10:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:10:35.061938564 +0000 UTC m=+941.822694465" watchObservedRunningTime="2026-10-30 22:10:35.063146638 +0000 UTC m=+941.823902527"
Oct 30 22:10:36.007672 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:10:36.007643    6133 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw1-2e5a75ca98/isvc-canary-stop-predictor-7f5d58797c-7dm5f"
Oct 30 22:10:36.009006 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:10:36.008958    6133 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-2e5a75ca98/isvc-canary-stop-predictor-7f5d58797c-7dm5f" podUID="bb80ffe3-13a0-4765-aa4f-4e74eb4f0a42" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.23:8080: connect: connection refused"
Oct 30 22:10:37.011590 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:10:37.011541    6133 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-2e5a75ca98/isvc-canary-stop-predictor-7f5d58797c-7dm5f" podUID="bb80ffe3-13a0-4765-aa4f-4e74eb4f0a42" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.23:8080: connect: connection refused"
Oct 30 22:10:39.527691 ip-10-0-143-246 kubenswrapper[6133]: E1030 22:10:39.527654    6133 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6jppj" containerName="extractor"
Oct 30 22:10:40.714680 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:10:40.714638    6133 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7b6759dd5d/isvc-sklearn-graph-raw-3d5b0-predictor-65bcf48db8-gdcfk" podUID="31df3cd4-f8e0-4dee-821a-919d8702200f" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.21:8080: connect: connection refused"
Oct 30 22:10:40.801717 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:10:40.801677    6133 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7b6759dd5d/isvc-xgboost-graph-raw-3d5b0-predictor-5bc48dbc76-l4lq8" podUID="5e6ed395-7d35-4ec4-aef1-da4c4fe4959a" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.22:8080: connect: connection refused"
Oct 30 22:10:42.016529 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:10:42.016497    6133 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw1-2e5a75ca98/isvc-canary-stop-predictor-7f5d58797c-7dm5f"
Oct 30 22:10:42.017114 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:10:42.017087    6133 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-2e5a75ca98/isvc-canary-stop-predictor-7f5d58797c-7dm5f" podUID="bb80ffe3-13a0-4765-aa4f-4e74eb4f0a42" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.23:8080: connect: connection refused"
Oct 30 22:10:46.350795 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:10:46.350749    6133 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7b6759dd5d/isvc-sklearn-graph-raw-3d5b0-predictor-65bcf48db8-gdcfk" podUID="31df3cd4-f8e0-4dee-821a-919d8702200f" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.21:8080: connect: connection refused"
Oct 30 22:10:50.801051 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:10:50.800979    6133 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7b6759dd5d/isvc-xgboost-graph-raw-3d5b0-predictor-5bc48dbc76-l4lq8" podUID="5e6ed395-7d35-4ec4-aef1-da4c4fe4959a" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.22:8080: connect: connection refused"
Oct 30 22:10:52.017682 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:10:52.017643    6133 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-2e5a75ca98/isvc-canary-stop-predictor-7f5d58797c-7dm5f" podUID="bb80ffe3-13a0-4765-aa4f-4e74eb4f0a42" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.23:8080: connect: connection refused"
Oct 30 22:10:56.351200 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:10:56.351165    6133 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw0-7b6759dd5d/isvc-sklearn-graph-raw-3d5b0-predictor-65bcf48db8-gdcfk"
Oct 30 22:11:00.801624 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:11:00.801594    6133 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw0-7b6759dd5d/isvc-xgboost-graph-raw-3d5b0-predictor-5bc48dbc76-l4lq8"
Oct 30 22:11:02.017678 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:11:02.017628    6133 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-2e5a75ca98/isvc-canary-stop-predictor-7f5d58797c-7dm5f" podUID="bb80ffe3-13a0-4765-aa4f-4e74eb4f0a42" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.23:8080: connect: connection refused"
Oct 30 22:11:09.527712 ip-10-0-143-246 kubenswrapper[6133]: E1030 22:11:09.527671    6133 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6jppj" containerName="extractor"
Oct 30 22:11:12.017170 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:11:12.017120    6133 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-2e5a75ca98/isvc-canary-stop-predictor-7f5d58797c-7dm5f" podUID="bb80ffe3-13a0-4765-aa4f-4e74eb4f0a42" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.23:8080: connect: connection refused"
Oct 30 22:11:19.290940 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:11:19.290903    6133 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw0-7b6759dd5d/isvc-sklearn-graph-raw-3d5b0-predictor-65bcf48db8-gdcfk"]
Oct 30 22:11:19.291375 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:11:19.291170    6133 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw0-7b6759dd5d/isvc-sklearn-graph-raw-3d5b0-predictor-65bcf48db8-gdcfk" podUID="31df3cd4-f8e0-4dee-821a-919d8702200f" containerName="kserve-container" containerID="cri-o://028db74615dc5d0853b4ddd65300c5bdca8572edb5dccc9bf8ad4f5ddac41c3f" gracePeriod=30
Oct 30 22:11:19.291375 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:11:19.291215    6133 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw0-7b6759dd5d/isvc-sklearn-graph-raw-3d5b0-predictor-65bcf48db8-gdcfk" podUID="31df3cd4-f8e0-4dee-821a-919d8702200f" containerName="kube-rbac-proxy" containerID="cri-o://73eac523ca16c17def4506a090fa9f6f1d2eaa98026fc8567850b832176e0295" gracePeriod=30
Oct 30 22:11:19.401172 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:11:19.401140    6133 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw0-7b6759dd5d/isvc-xgboost-graph-raw-3d5b0-predictor-5bc48dbc76-l4lq8"]
Oct 30 22:11:19.401399 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:11:19.401366    6133 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw0-7b6759dd5d/isvc-xgboost-graph-raw-3d5b0-predictor-5bc48dbc76-l4lq8" podUID="5e6ed395-7d35-4ec4-aef1-da4c4fe4959a" containerName="kserve-container" containerID="cri-o://62b1d91da93fc5b871a14d68c3e19d990b441b1caa1fcb1ad1fb6961bba43df1" gracePeriod=30
Oct 30 22:11:19.401494 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:11:19.401412    6133 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw0-7b6759dd5d/isvc-xgboost-graph-raw-3d5b0-predictor-5bc48dbc76-l4lq8" podUID="5e6ed395-7d35-4ec4-aef1-da4c4fe4959a" containerName="kube-rbac-proxy" containerID="cri-o://e9a761c8fe0e492c507324bd6c4e75dd1cd86f5b111f5b251bd86889f88c984a" gracePeriod=30
Oct 30 22:11:20.182165 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:11:20.182130    6133 generic.go:356] "Generic (PLEG): container finished" podID="31df3cd4-f8e0-4dee-821a-919d8702200f" containerID="73eac523ca16c17def4506a090fa9f6f1d2eaa98026fc8567850b832176e0295" exitCode=0
Oct 30 22:11:20.182374 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:11:20.182202    6133 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-7b6759dd5d/isvc-sklearn-graph-raw-3d5b0-predictor-65bcf48db8-gdcfk" event={"ID":"31df3cd4-f8e0-4dee-821a-919d8702200f","Type":"ContainerDied","Data":"73eac523ca16c17def4506a090fa9f6f1d2eaa98026fc8567850b832176e0295"}
Oct 30 22:11:20.184445 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:11:20.184422    6133 generic.go:356] "Generic (PLEG): container finished" podID="5e6ed395-7d35-4ec4-aef1-da4c4fe4959a" containerID="e9a761c8fe0e492c507324bd6c4e75dd1cd86f5b111f5b251bd86889f88c984a" exitCode=0
Oct 30 22:11:20.184535 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:11:20.184468    6133 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-7b6759dd5d/isvc-xgboost-graph-raw-3d5b0-predictor-5bc48dbc76-l4lq8" event={"ID":"5e6ed395-7d35-4ec4-aef1-da4c4fe4959a","Type":"ContainerDied","Data":"e9a761c8fe0e492c507324bd6c4e75dd1cd86f5b111f5b251bd86889f88c984a"}
Oct 30 22:11:20.709096 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:11:20.709054    6133 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7b6759dd5d/isvc-sklearn-graph-raw-3d5b0-predictor-65bcf48db8-gdcfk" podUID="31df3cd4-f8e0-4dee-821a-919d8702200f" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.21:8643/healthz\": dial tcp 10.134.0.21:8643: connect: connection refused"
Oct 30 22:11:20.796698 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:11:20.796655    6133 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7b6759dd5d/isvc-xgboost-graph-raw-3d5b0-predictor-5bc48dbc76-l4lq8" podUID="5e6ed395-7d35-4ec4-aef1-da4c4fe4959a" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.22:8643/healthz\": dial tcp 10.134.0.22:8643: connect: connection refused"
Oct 30 22:11:20.802067 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:11:20.802038    6133 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7b6759dd5d/isvc-xgboost-graph-raw-3d5b0-predictor-5bc48dbc76-l4lq8" podUID="5e6ed395-7d35-4ec4-aef1-da4c4fe4959a" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.22:8080: connect: connection refused"
Oct 30 22:11:22.017531 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:11:22.017493    6133 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-2e5a75ca98/isvc-canary-stop-predictor-7f5d58797c-7dm5f" podUID="bb80ffe3-13a0-4765-aa4f-4e74eb4f0a42" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.23:8080: connect: connection refused"
Oct 30 22:11:23.849489 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:11:23.849459    6133 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-7b6759dd5d/isvc-xgboost-graph-raw-3d5b0-predictor-5bc48dbc76-l4lq8"
Oct 30 22:11:24.025038 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:11:24.024967    6133 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/5e6ed395-7d35-4ec4-aef1-da4c4fe4959a-cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/5e6ed395-7d35-4ec4-aef1-da4c4fe4959a-cabundle-cert\") pod \"5e6ed395-7d35-4ec4-aef1-da4c4fe4959a\" (UID: \"5e6ed395-7d35-4ec4-aef1-da4c4fe4959a\") "
Oct 30 22:11:24.025237 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:11:24.025075    6133 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/5e6ed395-7d35-4ec4-aef1-da4c4fe4959a-proxy-tls\" (UniqueName: \"kubernetes.io/secret/5e6ed395-7d35-4ec4-aef1-da4c4fe4959a-proxy-tls\") pod \"5e6ed395-7d35-4ec4-aef1-da4c4fe4959a\" (UID: \"5e6ed395-7d35-4ec4-aef1-da4c4fe4959a\") "
Oct 30 22:11:24.025237 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:11:24.025098    6133 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5e6ed395-7d35-4ec4-aef1-da4c4fe4959a-kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/5e6ed395-7d35-4ec4-aef1-da4c4fe4959a-kserve-provision-location\") pod \"5e6ed395-7d35-4ec4-aef1-da4c4fe4959a\" (UID: \"5e6ed395-7d35-4ec4-aef1-da4c4fe4959a\") "
Oct 30 22:11:24.025237 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:11:24.025122    6133 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/5e6ed395-7d35-4ec4-aef1-da4c4fe4959a-kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/5e6ed395-7d35-4ec4-aef1-da4c4fe4959a-kube-rbac-proxy-sar-config\") pod \"5e6ed395-7d35-4ec4-aef1-da4c4fe4959a\" (UID: \"5e6ed395-7d35-4ec4-aef1-da4c4fe4959a\") "
Oct 30 22:11:24.025237 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:11:24.025144    6133 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5e6ed395-7d35-4ec4-aef1-da4c4fe4959a-kube-api-access-9jlvr\" (UniqueName: \"kubernetes.io/projected/5e6ed395-7d35-4ec4-aef1-da4c4fe4959a-kube-api-access-9jlvr\") pod \"5e6ed395-7d35-4ec4-aef1-da4c4fe4959a\" (UID: \"5e6ed395-7d35-4ec4-aef1-da4c4fe4959a\") "
Oct 30 22:11:24.025470 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:11:24.025405    6133 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5e6ed395-7d35-4ec4-aef1-da4c4fe4959a-cabundle-cert" pod "5e6ed395-7d35-4ec4-aef1-da4c4fe4959a" (UID: "5e6ed395-7d35-4ec4-aef1-da4c4fe4959a"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:11:24.025470 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:11:24.025455    6133 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5e6ed395-7d35-4ec4-aef1-da4c4fe4959a-kube-rbac-proxy-sar-config" pod "5e6ed395-7d35-4ec4-aef1-da4c4fe4959a" (UID: "5e6ed395-7d35-4ec4-aef1-da4c4fe4959a"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:11:24.025470 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:11:24.025456    6133 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5e6ed395-7d35-4ec4-aef1-da4c4fe4959a-kserve-provision-location" pod "5e6ed395-7d35-4ec4-aef1-da4c4fe4959a" (UID: "5e6ed395-7d35-4ec4-aef1-da4c4fe4959a"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:11:24.027289 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:11:24.027258    6133 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5e6ed395-7d35-4ec4-aef1-da4c4fe4959a-kube-api-access-9jlvr" pod "5e6ed395-7d35-4ec4-aef1-da4c4fe4959a" (UID: "5e6ed395-7d35-4ec4-aef1-da4c4fe4959a"). InnerVolumeSpecName "kube-api-access-9jlvr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:11:24.027377 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:11:24.027334    6133 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5e6ed395-7d35-4ec4-aef1-da4c4fe4959a-proxy-tls" pod "5e6ed395-7d35-4ec4-aef1-da4c4fe4959a" (UID: "5e6ed395-7d35-4ec4-aef1-da4c4fe4959a"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:11:24.126586 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:11:24.126547    6133 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/5e6ed395-7d35-4ec4-aef1-da4c4fe4959a-cabundle-cert\") on node \"ip-10-0-143-246.ec2.internal\" DevicePath \"\""
Oct 30 22:11:24.126586 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:11:24.126580    6133 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/5e6ed395-7d35-4ec4-aef1-da4c4fe4959a-proxy-tls\") on node \"ip-10-0-143-246.ec2.internal\" DevicePath \"\""
Oct 30 22:11:24.126586 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:11:24.126590    6133 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/5e6ed395-7d35-4ec4-aef1-da4c4fe4959a-kserve-provision-location\") on node \"ip-10-0-143-246.ec2.internal\" DevicePath \"\""
Oct 30 22:11:24.126812 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:11:24.126602    6133 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/5e6ed395-7d35-4ec4-aef1-da4c4fe4959a-kube-rbac-proxy-sar-config\") on node \"ip-10-0-143-246.ec2.internal\" DevicePath \"\""
Oct 30 22:11:24.126812 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:11:24.126611    6133 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9jlvr\" (UniqueName: \"kubernetes.io/projected/5e6ed395-7d35-4ec4-aef1-da4c4fe4959a-kube-api-access-9jlvr\") on node \"ip-10-0-143-246.ec2.internal\" DevicePath \"\""
Oct 30 22:11:24.203317 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:11:24.203284    6133 generic.go:356] "Generic (PLEG): container finished" podID="5e6ed395-7d35-4ec4-aef1-da4c4fe4959a" containerID="62b1d91da93fc5b871a14d68c3e19d990b441b1caa1fcb1ad1fb6961bba43df1" exitCode=0
Oct 30 22:11:24.203489 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:11:24.203342    6133 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-7b6759dd5d/isvc-xgboost-graph-raw-3d5b0-predictor-5bc48dbc76-l4lq8"
Oct 30 22:11:24.203489 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:11:24.203330    6133 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-7b6759dd5d/isvc-xgboost-graph-raw-3d5b0-predictor-5bc48dbc76-l4lq8" event={"ID":"5e6ed395-7d35-4ec4-aef1-da4c4fe4959a","Type":"ContainerDied","Data":"62b1d91da93fc5b871a14d68c3e19d990b441b1caa1fcb1ad1fb6961bba43df1"}
Oct 30 22:11:24.203489 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:11:24.203484    6133 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-7b6759dd5d/isvc-xgboost-graph-raw-3d5b0-predictor-5bc48dbc76-l4lq8" event={"ID":"5e6ed395-7d35-4ec4-aef1-da4c4fe4959a","Type":"ContainerDied","Data":"b8017acd9dea8b5d4efd1537081f71eb3dbcc4e1977c4968c8ae2dcee42e5782"}
Oct 30 22:11:24.203653 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:11:24.203523    6133 scope.go:122] "RemoveContainer" containerID="e9a761c8fe0e492c507324bd6c4e75dd1cd86f5b111f5b251bd86889f88c984a"
Oct 30 22:11:24.211774 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:11:24.211757    6133 scope.go:122] "RemoveContainer" containerID="62b1d91da93fc5b871a14d68c3e19d990b441b1caa1fcb1ad1fb6961bba43df1"
Oct 30 22:11:24.219209 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:11:24.219191    6133 scope.go:122] "RemoveContainer" containerID="24b996686d83786eea0c07ccfb8e5dcfa136a3fadf75a9e5f7f790362aec788c"
Oct 30 22:11:24.226784 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:11:24.226766    6133 scope.go:122] "RemoveContainer" containerID="e9a761c8fe0e492c507324bd6c4e75dd1cd86f5b111f5b251bd86889f88c984a"
Oct 30 22:11:24.227046 ip-10-0-143-246 kubenswrapper[6133]: E1030 22:11:24.227026    6133 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e9a761c8fe0e492c507324bd6c4e75dd1cd86f5b111f5b251bd86889f88c984a\": container with ID starting with e9a761c8fe0e492c507324bd6c4e75dd1cd86f5b111f5b251bd86889f88c984a not found: ID does not exist" containerID="e9a761c8fe0e492c507324bd6c4e75dd1cd86f5b111f5b251bd86889f88c984a"
Oct 30 22:11:24.227190 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:11:24.227054    6133 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e9a761c8fe0e492c507324bd6c4e75dd1cd86f5b111f5b251bd86889f88c984a"} err="failed to get container status \"e9a761c8fe0e492c507324bd6c4e75dd1cd86f5b111f5b251bd86889f88c984a\": rpc error: code = NotFound desc = could not find container \"e9a761c8fe0e492c507324bd6c4e75dd1cd86f5b111f5b251bd86889f88c984a\": container with ID starting with e9a761c8fe0e492c507324bd6c4e75dd1cd86f5b111f5b251bd86889f88c984a not found: ID does not exist"
Oct 30 22:11:24.227190 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:11:24.227073    6133 scope.go:122] "RemoveContainer" containerID="62b1d91da93fc5b871a14d68c3e19d990b441b1caa1fcb1ad1fb6961bba43df1"
Oct 30 22:11:24.227338 ip-10-0-143-246 kubenswrapper[6133]: E1030 22:11:24.227320    6133 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"62b1d91da93fc5b871a14d68c3e19d990b441b1caa1fcb1ad1fb6961bba43df1\": container with ID starting with 62b1d91da93fc5b871a14d68c3e19d990b441b1caa1fcb1ad1fb6961bba43df1 not found: ID does not exist" containerID="62b1d91da93fc5b871a14d68c3e19d990b441b1caa1fcb1ad1fb6961bba43df1"
Oct 30 22:11:24.227377 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:11:24.227346    6133 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"62b1d91da93fc5b871a14d68c3e19d990b441b1caa1fcb1ad1fb6961bba43df1"} err="failed to get container status \"62b1d91da93fc5b871a14d68c3e19d990b441b1caa1fcb1ad1fb6961bba43df1\": rpc error: code = NotFound desc = could not find container \"62b1d91da93fc5b871a14d68c3e19d990b441b1caa1fcb1ad1fb6961bba43df1\": container with ID starting with 62b1d91da93fc5b871a14d68c3e19d990b441b1caa1fcb1ad1fb6961bba43df1 not found: ID does not exist"
Oct 30 22:11:24.227377 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:11:24.227361    6133 scope.go:122] "RemoveContainer" containerID="24b996686d83786eea0c07ccfb8e5dcfa136a3fadf75a9e5f7f790362aec788c"
Oct 30 22:11:24.227624 ip-10-0-143-246 kubenswrapper[6133]: E1030 22:11:24.227607    6133 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"24b996686d83786eea0c07ccfb8e5dcfa136a3fadf75a9e5f7f790362aec788c\": container with ID starting with 24b996686d83786eea0c07ccfb8e5dcfa136a3fadf75a9e5f7f790362aec788c not found: ID does not exist" containerID="24b996686d83786eea0c07ccfb8e5dcfa136a3fadf75a9e5f7f790362aec788c"
Oct 30 22:11:24.227685 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:11:24.227632    6133 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"24b996686d83786eea0c07ccfb8e5dcfa136a3fadf75a9e5f7f790362aec788c"} err="failed to get container status \"24b996686d83786eea0c07ccfb8e5dcfa136a3fadf75a9e5f7f790362aec788c\": rpc error: code = NotFound desc = could not find container \"24b996686d83786eea0c07ccfb8e5dcfa136a3fadf75a9e5f7f790362aec788c\": container with ID starting with 24b996686d83786eea0c07ccfb8e5dcfa136a3fadf75a9e5f7f790362aec788c not found: ID does not exist"
Oct 30 22:11:24.259076 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:11:24.259038    6133 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw0-7b6759dd5d/isvc-xgboost-graph-raw-3d5b0-predictor-5bc48dbc76-l4lq8"]
Oct 30 22:11:24.271474 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:11:24.271443    6133 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-gw0-7b6759dd5d/isvc-xgboost-graph-raw-3d5b0-predictor-5bc48dbc76-l4lq8"]
Oct 30 22:11:24.624516 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:11:24.624489    6133 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-7b6759dd5d/isvc-sklearn-graph-raw-3d5b0-predictor-65bcf48db8-gdcfk"
Oct 30 22:11:24.731768 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:11:24.731683    6133 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/31df3cd4-f8e0-4dee-821a-919d8702200f-kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/31df3cd4-f8e0-4dee-821a-919d8702200f-kube-rbac-proxy-sar-config\") pod \"31df3cd4-f8e0-4dee-821a-919d8702200f\" (UID: \"31df3cd4-f8e0-4dee-821a-919d8702200f\") "
Oct 30 22:11:24.731768 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:11:24.731725    6133 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/31df3cd4-f8e0-4dee-821a-919d8702200f-kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/31df3cd4-f8e0-4dee-821a-919d8702200f-kserve-provision-location\") pod \"31df3cd4-f8e0-4dee-821a-919d8702200f\" (UID: \"31df3cd4-f8e0-4dee-821a-919d8702200f\") "
Oct 30 22:11:24.731768 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:11:24.731768    6133 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/31df3cd4-f8e0-4dee-821a-919d8702200f-proxy-tls\" (UniqueName: \"kubernetes.io/secret/31df3cd4-f8e0-4dee-821a-919d8702200f-proxy-tls\") pod \"31df3cd4-f8e0-4dee-821a-919d8702200f\" (UID: \"31df3cd4-f8e0-4dee-821a-919d8702200f\") "
Oct 30 22:11:24.732028 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:11:24.731804    6133 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/31df3cd4-f8e0-4dee-821a-919d8702200f-cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/31df3cd4-f8e0-4dee-821a-919d8702200f-cabundle-cert\") pod \"31df3cd4-f8e0-4dee-821a-919d8702200f\" (UID: \"31df3cd4-f8e0-4dee-821a-919d8702200f\") "
Oct 30 22:11:24.732028 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:11:24.731872    6133 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/31df3cd4-f8e0-4dee-821a-919d8702200f-kube-api-access-jq8wj\" (UniqueName: \"kubernetes.io/projected/31df3cd4-f8e0-4dee-821a-919d8702200f-kube-api-access-jq8wj\") pod \"31df3cd4-f8e0-4dee-821a-919d8702200f\" (UID: \"31df3cd4-f8e0-4dee-821a-919d8702200f\") "
Oct 30 22:11:24.732219 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:11:24.732126    6133 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/31df3cd4-f8e0-4dee-821a-919d8702200f-kserve-provision-location" pod "31df3cd4-f8e0-4dee-821a-919d8702200f" (UID: "31df3cd4-f8e0-4dee-821a-919d8702200f"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:11:24.732270 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:11:24.732221    6133 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/31df3cd4-f8e0-4dee-821a-919d8702200f-kube-rbac-proxy-sar-config" pod "31df3cd4-f8e0-4dee-821a-919d8702200f" (UID: "31df3cd4-f8e0-4dee-821a-919d8702200f"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:11:24.732311 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:11:24.732283    6133 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/31df3cd4-f8e0-4dee-821a-919d8702200f-cabundle-cert" pod "31df3cd4-f8e0-4dee-821a-919d8702200f" (UID: "31df3cd4-f8e0-4dee-821a-919d8702200f"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:11:24.733944 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:11:24.733922    6133 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/31df3cd4-f8e0-4dee-821a-919d8702200f-proxy-tls" pod "31df3cd4-f8e0-4dee-821a-919d8702200f" (UID: "31df3cd4-f8e0-4dee-821a-919d8702200f"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:11:24.734037 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:11:24.733960    6133 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/31df3cd4-f8e0-4dee-821a-919d8702200f-kube-api-access-jq8wj" pod "31df3cd4-f8e0-4dee-821a-919d8702200f" (UID: "31df3cd4-f8e0-4dee-821a-919d8702200f"). InnerVolumeSpecName "kube-api-access-jq8wj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:11:24.832952 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:11:24.832912    6133 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/31df3cd4-f8e0-4dee-821a-919d8702200f-proxy-tls\") on node \"ip-10-0-143-246.ec2.internal\" DevicePath \"\""
Oct 30 22:11:24.832952 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:11:24.832946    6133 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/31df3cd4-f8e0-4dee-821a-919d8702200f-cabundle-cert\") on node \"ip-10-0-143-246.ec2.internal\" DevicePath \"\""
Oct 30 22:11:24.832952 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:11:24.832955    6133 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jq8wj\" (UniqueName: \"kubernetes.io/projected/31df3cd4-f8e0-4dee-821a-919d8702200f-kube-api-access-jq8wj\") on node \"ip-10-0-143-246.ec2.internal\" DevicePath \"\""
Oct 30 22:11:24.832952 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:11:24.832965    6133 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/31df3cd4-f8e0-4dee-821a-919d8702200f-kube-rbac-proxy-sar-config\") on node \"ip-10-0-143-246.ec2.internal\" DevicePath \"\""
Oct 30 22:11:24.833244 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:11:24.832974    6133 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/31df3cd4-f8e0-4dee-821a-919d8702200f-kserve-provision-location\") on node \"ip-10-0-143-246.ec2.internal\" DevicePath \"\""
Oct 30 22:11:25.209658 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:11:25.209622    6133 generic.go:356] "Generic (PLEG): container finished" podID="31df3cd4-f8e0-4dee-821a-919d8702200f" containerID="028db74615dc5d0853b4ddd65300c5bdca8572edb5dccc9bf8ad4f5ddac41c3f" exitCode=0
Oct 30 22:11:25.210102 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:11:25.209689    6133 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-7b6759dd5d/isvc-sklearn-graph-raw-3d5b0-predictor-65bcf48db8-gdcfk" event={"ID":"31df3cd4-f8e0-4dee-821a-919d8702200f","Type":"ContainerDied","Data":"028db74615dc5d0853b4ddd65300c5bdca8572edb5dccc9bf8ad4f5ddac41c3f"}
Oct 30 22:11:25.210102 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:11:25.209738    6133 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-7b6759dd5d/isvc-sklearn-graph-raw-3d5b0-predictor-65bcf48db8-gdcfk" event={"ID":"31df3cd4-f8e0-4dee-821a-919d8702200f","Type":"ContainerDied","Data":"51d53d9daa9fa7f2904ca99cdf1480bd3e7007b3edf99d9ce0423a378e09e3fd"}
Oct 30 22:11:25.210102 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:11:25.209692    6133 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-7b6759dd5d/isvc-sklearn-graph-raw-3d5b0-predictor-65bcf48db8-gdcfk"
Oct 30 22:11:25.210102 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:11:25.209770    6133 scope.go:122] "RemoveContainer" containerID="73eac523ca16c17def4506a090fa9f6f1d2eaa98026fc8567850b832176e0295"
Oct 30 22:11:25.217723 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:11:25.217705    6133 scope.go:122] "RemoveContainer" containerID="028db74615dc5d0853b4ddd65300c5bdca8572edb5dccc9bf8ad4f5ddac41c3f"
Oct 30 22:11:25.225064 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:11:25.225043    6133 scope.go:122] "RemoveContainer" containerID="f25c3e1543264ab4a356827484e62f495f33df5f8c06cb822bc2ecb4268892e5"
Oct 30 22:11:25.232281 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:11:25.232263    6133 scope.go:122] "RemoveContainer" containerID="73eac523ca16c17def4506a090fa9f6f1d2eaa98026fc8567850b832176e0295"
Oct 30 22:11:25.232549 ip-10-0-143-246 kubenswrapper[6133]: E1030 22:11:25.232522    6133 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"73eac523ca16c17def4506a090fa9f6f1d2eaa98026fc8567850b832176e0295\": container with ID starting with 73eac523ca16c17def4506a090fa9f6f1d2eaa98026fc8567850b832176e0295 not found: ID does not exist" containerID="73eac523ca16c17def4506a090fa9f6f1d2eaa98026fc8567850b832176e0295"
Oct 30 22:11:25.232603 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:11:25.232562    6133 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"73eac523ca16c17def4506a090fa9f6f1d2eaa98026fc8567850b832176e0295"} err="failed to get container status \"73eac523ca16c17def4506a090fa9f6f1d2eaa98026fc8567850b832176e0295\": rpc error: code = NotFound desc = could not find container \"73eac523ca16c17def4506a090fa9f6f1d2eaa98026fc8567850b832176e0295\": container with ID starting with 73eac523ca16c17def4506a090fa9f6f1d2eaa98026fc8567850b832176e0295 not found: ID does not exist"
Oct 30 22:11:25.232603 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:11:25.232582    6133 scope.go:122] "RemoveContainer" containerID="028db74615dc5d0853b4ddd65300c5bdca8572edb5dccc9bf8ad4f5ddac41c3f"
Oct 30 22:11:25.232807 ip-10-0-143-246 kubenswrapper[6133]: E1030 22:11:25.232790    6133 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"028db74615dc5d0853b4ddd65300c5bdca8572edb5dccc9bf8ad4f5ddac41c3f\": container with ID starting with 028db74615dc5d0853b4ddd65300c5bdca8572edb5dccc9bf8ad4f5ddac41c3f not found: ID does not exist" containerID="028db74615dc5d0853b4ddd65300c5bdca8572edb5dccc9bf8ad4f5ddac41c3f"
Oct 30 22:11:25.232854 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:11:25.232814    6133 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"028db74615dc5d0853b4ddd65300c5bdca8572edb5dccc9bf8ad4f5ddac41c3f"} err="failed to get container status \"028db74615dc5d0853b4ddd65300c5bdca8572edb5dccc9bf8ad4f5ddac41c3f\": rpc error: code = NotFound desc = could not find container \"028db74615dc5d0853b4ddd65300c5bdca8572edb5dccc9bf8ad4f5ddac41c3f\": container with ID starting with 028db74615dc5d0853b4ddd65300c5bdca8572edb5dccc9bf8ad4f5ddac41c3f not found: ID does not exist"
Oct 30 22:11:25.232854 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:11:25.232830    6133 scope.go:122] "RemoveContainer" containerID="f25c3e1543264ab4a356827484e62f495f33df5f8c06cb822bc2ecb4268892e5"
Oct 30 22:11:25.233049 ip-10-0-143-246 kubenswrapper[6133]: E1030 22:11:25.233031    6133 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f25c3e1543264ab4a356827484e62f495f33df5f8c06cb822bc2ecb4268892e5\": container with ID starting with f25c3e1543264ab4a356827484e62f495f33df5f8c06cb822bc2ecb4268892e5 not found: ID does not exist" containerID="f25c3e1543264ab4a356827484e62f495f33df5f8c06cb822bc2ecb4268892e5"
Oct 30 22:11:25.233099 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:11:25.233051    6133 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f25c3e1543264ab4a356827484e62f495f33df5f8c06cb822bc2ecb4268892e5"} err="failed to get container status \"f25c3e1543264ab4a356827484e62f495f33df5f8c06cb822bc2ecb4268892e5\": rpc error: code = NotFound desc = could not find container \"f25c3e1543264ab4a356827484e62f495f33df5f8c06cb822bc2ecb4268892e5\": container with ID starting with f25c3e1543264ab4a356827484e62f495f33df5f8c06cb822bc2ecb4268892e5 not found: ID does not exist"
Oct 30 22:11:25.277974 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:11:25.277942    6133 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw0-7b6759dd5d/isvc-sklearn-graph-raw-3d5b0-predictor-65bcf48db8-gdcfk"]
Oct 30 22:11:25.290355 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:11:25.290322    6133 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-gw0-7b6759dd5d/isvc-sklearn-graph-raw-3d5b0-predictor-65bcf48db8-gdcfk"]
Oct 30 22:11:25.355407 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:11:25.355373    6133 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="31df3cd4-f8e0-4dee-821a-919d8702200f" path="/var/lib/kubelet/pods/31df3cd4-f8e0-4dee-821a-919d8702200f/volumes"
Oct 30 22:11:25.355862 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:11:25.355849    6133 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5e6ed395-7d35-4ec4-aef1-da4c4fe4959a" path="/var/lib/kubelet/pods/5e6ed395-7d35-4ec4-aef1-da4c4fe4959a/volumes"
Oct 30 22:11:26.849439 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:11:26.849405    6133 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-gw0-7b6759dd5d/isvc-sklearn-graph-raw-hpa-a5a79-predictor-5f647f9b68-5t9lb"]
Oct 30 22:11:26.849833 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:11:26.849746    6133 cpu_manager.go:414] "containerMap: removing container" podUID="5e6ed395-7d35-4ec4-aef1-da4c4fe4959a" containerName="storage-initializer"
Oct 30 22:11:26.849833 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:11:26.849757    6133 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5e6ed395-7d35-4ec4-aef1-da4c4fe4959a" containerName="storage-initializer"
Oct 30 22:11:26.849833 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:11:26.849776    6133 cpu_manager.go:414] "containerMap: removing container" podUID="31df3cd4-f8e0-4dee-821a-919d8702200f" containerName="kube-rbac-proxy"
Oct 30 22:11:26.849833 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:11:26.849781    6133 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="31df3cd4-f8e0-4dee-821a-919d8702200f" containerName="kube-rbac-proxy"
Oct 30 22:11:26.849833 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:11:26.849791    6133 cpu_manager.go:414] "containerMap: removing container" podUID="5e6ed395-7d35-4ec4-aef1-da4c4fe4959a" containerName="kube-rbac-proxy"
Oct 30 22:11:26.849833 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:11:26.849796    6133 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5e6ed395-7d35-4ec4-aef1-da4c4fe4959a" containerName="kube-rbac-proxy"
Oct 30 22:11:26.849833 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:11:26.849809    6133 cpu_manager.go:414] "containerMap: removing container" podUID="31df3cd4-f8e0-4dee-821a-919d8702200f" containerName="storage-initializer"
Oct 30 22:11:26.849833 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:11:26.849814    6133 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="31df3cd4-f8e0-4dee-821a-919d8702200f" containerName="storage-initializer"
Oct 30 22:11:26.849833 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:11:26.849822    6133 cpu_manager.go:414] "containerMap: removing container" podUID="31df3cd4-f8e0-4dee-821a-919d8702200f" containerName="kserve-container"
Oct 30 22:11:26.849833 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:11:26.849827    6133 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="31df3cd4-f8e0-4dee-821a-919d8702200f" containerName="kserve-container"
Oct 30 22:11:26.850150 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:11:26.849842    6133 cpu_manager.go:414] "containerMap: removing container" podUID="5e6ed395-7d35-4ec4-aef1-da4c4fe4959a" containerName="kserve-container"
Oct 30 22:11:26.850150 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:11:26.849847    6133 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5e6ed395-7d35-4ec4-aef1-da4c4fe4959a" containerName="kserve-container"
Oct 30 22:11:26.850150 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:11:26.849898    6133 memory_manager.go:365] "RemoveStaleState removing state" podUID="5e6ed395-7d35-4ec4-aef1-da4c4fe4959a" containerName="kserve-container"
Oct 30 22:11:26.850150 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:11:26.849908    6133 memory_manager.go:365] "RemoveStaleState removing state" podUID="31df3cd4-f8e0-4dee-821a-919d8702200f" containerName="kube-rbac-proxy"
Oct 30 22:11:26.850150 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:11:26.849914    6133 memory_manager.go:365] "RemoveStaleState removing state" podUID="5e6ed395-7d35-4ec4-aef1-da4c4fe4959a" containerName="kube-rbac-proxy"
Oct 30 22:11:26.850150 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:11:26.849919    6133 memory_manager.go:365] "RemoveStaleState removing state" podUID="31df3cd4-f8e0-4dee-821a-919d8702200f" containerName="kserve-container"
Oct 30 22:11:26.854860 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:11:26.854837    6133 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-7b6759dd5d/isvc-sklearn-graph-raw-hpa-a5a79-predictor-5f647f9b68-5t9lb"
Oct 30 22:11:26.885698 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:11:26.885655    6133 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw0-7b6759dd5d\"/\"isvc-sklearn-graph-raw-hpa-a5a79-kube-rbac-proxy-sar-config\""
Oct 30 22:11:26.885698 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:11:26.885691    6133 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw0-7b6759dd5d\"/\"kube-root-ca.crt\""
Oct 30 22:11:26.885950 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:11:26.885702    6133 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw0-7b6759dd5d\"/\"odh-kserve-custom-ca-bundle\""
Oct 30 22:11:26.885950 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:11:26.885790    6133 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw0-7b6759dd5d\"/\"openshift-service-ca.crt\""
Oct 30 22:11:26.885950 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:11:26.885902    6133 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw0-7b6759dd5d\"/\"isvc-sklearn-graph-raw-hpa-a5a79-predictor-serving-cert\""
Oct 30 22:11:26.885950 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:11:26.885922    6133 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw0-7b6759dd5d\"/\"seaweedfs-s3-creds\""
Oct 30 22:11:26.886217 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:11:26.886201    6133 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw0-7b6759dd5d\"/\"default-dockercfg-4b7xh\""
Oct 30 22:11:26.887557 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:11:26.887539    6133 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw0-7b6759dd5d/isvc-sklearn-graph-raw-hpa-a5a79-predictor-5f647f9b68-5t9lb"]
Oct 30 22:11:26.956462 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:11:26.956424    6133 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/b184588e-b671-4841-b1fe-ee453ee1116c-kserve-provision-location\") pod \"isvc-sklearn-graph-raw-hpa-a5a79-predictor-5f647f9b68-5t9lb\" (UID: \"b184588e-b671-4841-b1fe-ee453ee1116c\") " pod="e2e-gw0-7b6759dd5d/isvc-sklearn-graph-raw-hpa-a5a79-predictor-5f647f9b68-5t9lb"
Oct 30 22:11:26.956462 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:11:26.956461    6133 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/b184588e-b671-4841-b1fe-ee453ee1116c-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-graph-raw-hpa-a5a79-predictor-5f647f9b68-5t9lb\" (UID: \"b184588e-b671-4841-b1fe-ee453ee1116c\") " pod="e2e-gw0-7b6759dd5d/isvc-sklearn-graph-raw-hpa-a5a79-predictor-5f647f9b68-5t9lb"
Oct 30 22:11:26.956699 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:11:26.956484    6133 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/b184588e-b671-4841-b1fe-ee453ee1116c-proxy-tls\") pod \"isvc-sklearn-graph-raw-hpa-a5a79-predictor-5f647f9b68-5t9lb\" (UID: \"b184588e-b671-4841-b1fe-ee453ee1116c\") " pod="e2e-gw0-7b6759dd5d/isvc-sklearn-graph-raw-hpa-a5a79-predictor-5f647f9b68-5t9lb"
Oct 30 22:11:26.956699 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:11:26.956628    6133 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g2hm8\" (UniqueName: \"kubernetes.io/projected/b184588e-b671-4841-b1fe-ee453ee1116c-kube-api-access-g2hm8\") pod \"isvc-sklearn-graph-raw-hpa-a5a79-predictor-5f647f9b68-5t9lb\" (UID: \"b184588e-b671-4841-b1fe-ee453ee1116c\") " pod="e2e-gw0-7b6759dd5d/isvc-sklearn-graph-raw-hpa-a5a79-predictor-5f647f9b68-5t9lb"
Oct 30 22:11:26.956699 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:11:26.956671    6133 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/b184588e-b671-4841-b1fe-ee453ee1116c-cabundle-cert\") pod \"isvc-sklearn-graph-raw-hpa-a5a79-predictor-5f647f9b68-5t9lb\" (UID: \"b184588e-b671-4841-b1fe-ee453ee1116c\") " pod="e2e-gw0-7b6759dd5d/isvc-sklearn-graph-raw-hpa-a5a79-predictor-5f647f9b68-5t9lb"
Oct 30 22:11:27.057918 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:11:27.057884    6133 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/b184588e-b671-4841-b1fe-ee453ee1116c-kserve-provision-location\") pod \"isvc-sklearn-graph-raw-hpa-a5a79-predictor-5f647f9b68-5t9lb\" (UID: \"b184588e-b671-4841-b1fe-ee453ee1116c\") " pod="e2e-gw0-7b6759dd5d/isvc-sklearn-graph-raw-hpa-a5a79-predictor-5f647f9b68-5t9lb"
Oct 30 22:11:27.057918 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:11:27.057919    6133 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/b184588e-b671-4841-b1fe-ee453ee1116c-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-graph-raw-hpa-a5a79-predictor-5f647f9b68-5t9lb\" (UID: \"b184588e-b671-4841-b1fe-ee453ee1116c\") " pod="e2e-gw0-7b6759dd5d/isvc-sklearn-graph-raw-hpa-a5a79-predictor-5f647f9b68-5t9lb"
Oct 30 22:11:27.058222 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:11:27.057942    6133 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/b184588e-b671-4841-b1fe-ee453ee1116c-proxy-tls\") pod \"isvc-sklearn-graph-raw-hpa-a5a79-predictor-5f647f9b68-5t9lb\" (UID: \"b184588e-b671-4841-b1fe-ee453ee1116c\") " pod="e2e-gw0-7b6759dd5d/isvc-sklearn-graph-raw-hpa-a5a79-predictor-5f647f9b68-5t9lb"
Oct 30 22:11:27.058222 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:11:27.058013    6133 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g2hm8\" (UniqueName: \"kubernetes.io/projected/b184588e-b671-4841-b1fe-ee453ee1116c-kube-api-access-g2hm8\") pod \"isvc-sklearn-graph-raw-hpa-a5a79-predictor-5f647f9b68-5t9lb\" (UID: \"b184588e-b671-4841-b1fe-ee453ee1116c\") " pod="e2e-gw0-7b6759dd5d/isvc-sklearn-graph-raw-hpa-a5a79-predictor-5f647f9b68-5t9lb"
Oct 30 22:11:27.058222 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:11:27.058055    6133 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/b184588e-b671-4841-b1fe-ee453ee1116c-cabundle-cert\") pod \"isvc-sklearn-graph-raw-hpa-a5a79-predictor-5f647f9b68-5t9lb\" (UID: \"b184588e-b671-4841-b1fe-ee453ee1116c\") " pod="e2e-gw0-7b6759dd5d/isvc-sklearn-graph-raw-hpa-a5a79-predictor-5f647f9b68-5t9lb"
Oct 30 22:11:27.058464 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:11:27.058425    6133 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/b184588e-b671-4841-b1fe-ee453ee1116c-kserve-provision-location\") pod \"isvc-sklearn-graph-raw-hpa-a5a79-predictor-5f647f9b68-5t9lb\" (UID: \"b184588e-b671-4841-b1fe-ee453ee1116c\") " pod="e2e-gw0-7b6759dd5d/isvc-sklearn-graph-raw-hpa-a5a79-predictor-5f647f9b68-5t9lb"
Oct 30 22:11:27.058794 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:11:27.058773    6133 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/b184588e-b671-4841-b1fe-ee453ee1116c-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-graph-raw-hpa-a5a79-predictor-5f647f9b68-5t9lb\" (UID: \"b184588e-b671-4841-b1fe-ee453ee1116c\") " pod="e2e-gw0-7b6759dd5d/isvc-sklearn-graph-raw-hpa-a5a79-predictor-5f647f9b68-5t9lb"
Oct 30 22:11:27.058921 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:11:27.058872    6133 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/b184588e-b671-4841-b1fe-ee453ee1116c-cabundle-cert\") pod \"isvc-sklearn-graph-raw-hpa-a5a79-predictor-5f647f9b68-5t9lb\" (UID: \"b184588e-b671-4841-b1fe-ee453ee1116c\") " pod="e2e-gw0-7b6759dd5d/isvc-sklearn-graph-raw-hpa-a5a79-predictor-5f647f9b68-5t9lb"
Oct 30 22:11:27.060768 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:11:27.060749    6133 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/b184588e-b671-4841-b1fe-ee453ee1116c-proxy-tls\") pod \"isvc-sklearn-graph-raw-hpa-a5a79-predictor-5f647f9b68-5t9lb\" (UID: \"b184588e-b671-4841-b1fe-ee453ee1116c\") " pod="e2e-gw0-7b6759dd5d/isvc-sklearn-graph-raw-hpa-a5a79-predictor-5f647f9b68-5t9lb"
Oct 30 22:11:27.082573 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:11:27.082542    6133 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g2hm8\" (UniqueName: \"kubernetes.io/projected/b184588e-b671-4841-b1fe-ee453ee1116c-kube-api-access-g2hm8\") pod \"isvc-sklearn-graph-raw-hpa-a5a79-predictor-5f647f9b68-5t9lb\" (UID: \"b184588e-b671-4841-b1fe-ee453ee1116c\") " pod="e2e-gw0-7b6759dd5d/isvc-sklearn-graph-raw-hpa-a5a79-predictor-5f647f9b68-5t9lb"
Oct 30 22:11:27.172587 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:11:27.172485    6133 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-7b6759dd5d/isvc-sklearn-graph-raw-hpa-a5a79-predictor-5f647f9b68-5t9lb"
Oct 30 22:11:27.225426 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:11:27.225396    6133 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-gw0-7b6759dd5d/isvc-xgboost-graph-raw-hpa-a5a79-predictor-869d547bd8-f9hpm"]
Oct 30 22:11:27.229873 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:11:27.229838    6133 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-7b6759dd5d/isvc-xgboost-graph-raw-hpa-a5a79-predictor-869d547bd8-f9hpm"
Oct 30 22:11:27.251407 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:11:27.251362    6133 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw0-7b6759dd5d\"/\"isvc-xgboost-graph-raw-hpa-a5a79-kube-rbac-proxy-sar-config\""
Oct 30 22:11:27.252598 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:11:27.252562    6133 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw0-7b6759dd5d\"/\"isvc-xgboost-graph-raw-hpa-a5a79-predictor-serving-cert\""
Oct 30 22:11:27.257052 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:11:27.256960    6133 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw0-7b6759dd5d/isvc-xgboost-graph-raw-hpa-a5a79-predictor-869d547bd8-f9hpm"]
Oct 30 22:11:27.360241 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:11:27.360207    6133 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/fbd531c4-ba6f-4632-b6f2-d1fd9dead011-kube-rbac-proxy-sar-config\") pod \"isvc-xgboost-graph-raw-hpa-a5a79-predictor-869d547bd8-f9hpm\" (UID: \"fbd531c4-ba6f-4632-b6f2-d1fd9dead011\") " pod="e2e-gw0-7b6759dd5d/isvc-xgboost-graph-raw-hpa-a5a79-predictor-869d547bd8-f9hpm"
Oct 30 22:11:27.360415 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:11:27.360246    6133 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/fbd531c4-ba6f-4632-b6f2-d1fd9dead011-kserve-provision-location\") pod \"isvc-xgboost-graph-raw-hpa-a5a79-predictor-869d547bd8-f9hpm\" (UID: \"fbd531c4-ba6f-4632-b6f2-d1fd9dead011\") " pod="e2e-gw0-7b6759dd5d/isvc-xgboost-graph-raw-hpa-a5a79-predictor-869d547bd8-f9hpm"
Oct 30 22:11:27.360415 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:11:27.360269    6133 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d4t4g\" (UniqueName: \"kubernetes.io/projected/fbd531c4-ba6f-4632-b6f2-d1fd9dead011-kube-api-access-d4t4g\") pod \"isvc-xgboost-graph-raw-hpa-a5a79-predictor-869d547bd8-f9hpm\" (UID: \"fbd531c4-ba6f-4632-b6f2-d1fd9dead011\") " pod="e2e-gw0-7b6759dd5d/isvc-xgboost-graph-raw-hpa-a5a79-predictor-869d547bd8-f9hpm"
Oct 30 22:11:27.360415 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:11:27.360371    6133 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/fbd531c4-ba6f-4632-b6f2-d1fd9dead011-cabundle-cert\") pod \"isvc-xgboost-graph-raw-hpa-a5a79-predictor-869d547bd8-f9hpm\" (UID: \"fbd531c4-ba6f-4632-b6f2-d1fd9dead011\") " pod="e2e-gw0-7b6759dd5d/isvc-xgboost-graph-raw-hpa-a5a79-predictor-869d547bd8-f9hpm"
Oct 30 22:11:27.360518 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:11:27.360418    6133 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/fbd531c4-ba6f-4632-b6f2-d1fd9dead011-proxy-tls\") pod \"isvc-xgboost-graph-raw-hpa-a5a79-predictor-869d547bd8-f9hpm\" (UID: \"fbd531c4-ba6f-4632-b6f2-d1fd9dead011\") " pod="e2e-gw0-7b6759dd5d/isvc-xgboost-graph-raw-hpa-a5a79-predictor-869d547bd8-f9hpm"
Oct 30 22:11:27.461647 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:11:27.461569    6133 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/fbd531c4-ba6f-4632-b6f2-d1fd9dead011-proxy-tls\") pod \"isvc-xgboost-graph-raw-hpa-a5a79-predictor-869d547bd8-f9hpm\" (UID: \"fbd531c4-ba6f-4632-b6f2-d1fd9dead011\") " pod="e2e-gw0-7b6759dd5d/isvc-xgboost-graph-raw-hpa-a5a79-predictor-869d547bd8-f9hpm"
Oct 30 22:11:27.461647 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:11:27.461639    6133 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/fbd531c4-ba6f-4632-b6f2-d1fd9dead011-kube-rbac-proxy-sar-config\") pod \"isvc-xgboost-graph-raw-hpa-a5a79-predictor-869d547bd8-f9hpm\" (UID: \"fbd531c4-ba6f-4632-b6f2-d1fd9dead011\") " pod="e2e-gw0-7b6759dd5d/isvc-xgboost-graph-raw-hpa-a5a79-predictor-869d547bd8-f9hpm"
Oct 30 22:11:27.461860 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:11:27.461675    6133 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/fbd531c4-ba6f-4632-b6f2-d1fd9dead011-kserve-provision-location\") pod \"isvc-xgboost-graph-raw-hpa-a5a79-predictor-869d547bd8-f9hpm\" (UID: \"fbd531c4-ba6f-4632-b6f2-d1fd9dead011\") " pod="e2e-gw0-7b6759dd5d/isvc-xgboost-graph-raw-hpa-a5a79-predictor-869d547bd8-f9hpm"
Oct 30 22:11:27.461860 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:11:27.461703    6133 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d4t4g\" (UniqueName: \"kubernetes.io/projected/fbd531c4-ba6f-4632-b6f2-d1fd9dead011-kube-api-access-d4t4g\") pod \"isvc-xgboost-graph-raw-hpa-a5a79-predictor-869d547bd8-f9hpm\" (UID: \"fbd531c4-ba6f-4632-b6f2-d1fd9dead011\") " pod="e2e-gw0-7b6759dd5d/isvc-xgboost-graph-raw-hpa-a5a79-predictor-869d547bd8-f9hpm"
Oct 30 22:11:27.461860 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:11:27.461729    6133 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/fbd531c4-ba6f-4632-b6f2-d1fd9dead011-cabundle-cert\") pod \"isvc-xgboost-graph-raw-hpa-a5a79-predictor-869d547bd8-f9hpm\" (UID: \"fbd531c4-ba6f-4632-b6f2-d1fd9dead011\") " pod="e2e-gw0-7b6759dd5d/isvc-xgboost-graph-raw-hpa-a5a79-predictor-869d547bd8-f9hpm"
Oct 30 22:11:27.462208 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:11:27.462182    6133 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/fbd531c4-ba6f-4632-b6f2-d1fd9dead011-kserve-provision-location\") pod \"isvc-xgboost-graph-raw-hpa-a5a79-predictor-869d547bd8-f9hpm\" (UID: \"fbd531c4-ba6f-4632-b6f2-d1fd9dead011\") " pod="e2e-gw0-7b6759dd5d/isvc-xgboost-graph-raw-hpa-a5a79-predictor-869d547bd8-f9hpm"
Oct 30 22:11:27.462510 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:11:27.462488    6133 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/fbd531c4-ba6f-4632-b6f2-d1fd9dead011-cabundle-cert\") pod \"isvc-xgboost-graph-raw-hpa-a5a79-predictor-869d547bd8-f9hpm\" (UID: \"fbd531c4-ba6f-4632-b6f2-d1fd9dead011\") " pod="e2e-gw0-7b6759dd5d/isvc-xgboost-graph-raw-hpa-a5a79-predictor-869d547bd8-f9hpm"
Oct 30 22:11:27.462510 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:11:27.462501    6133 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/fbd531c4-ba6f-4632-b6f2-d1fd9dead011-kube-rbac-proxy-sar-config\") pod \"isvc-xgboost-graph-raw-hpa-a5a79-predictor-869d547bd8-f9hpm\" (UID: \"fbd531c4-ba6f-4632-b6f2-d1fd9dead011\") " pod="e2e-gw0-7b6759dd5d/isvc-xgboost-graph-raw-hpa-a5a79-predictor-869d547bd8-f9hpm"
Oct 30 22:11:27.464044 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:11:27.464022    6133 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/fbd531c4-ba6f-4632-b6f2-d1fd9dead011-proxy-tls\") pod \"isvc-xgboost-graph-raw-hpa-a5a79-predictor-869d547bd8-f9hpm\" (UID: \"fbd531c4-ba6f-4632-b6f2-d1fd9dead011\") " pod="e2e-gw0-7b6759dd5d/isvc-xgboost-graph-raw-hpa-a5a79-predictor-869d547bd8-f9hpm"
Oct 30 22:11:27.482573 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:11:27.482548    6133 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw0-7b6759dd5d/isvc-sklearn-graph-raw-hpa-a5a79-predictor-5f647f9b68-5t9lb"]
Oct 30 22:11:27.483912 ip-10-0-143-246 kubenswrapper[6133]: W1030 22:11:27.483884    6133 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb184588e_b671_4841_b1fe_ee453ee1116c.slice/crio-5e9bb39ce79f2a386f1a4bd1ed0f01ae29ebb7fe47d70330502d542886bb6914 WatchSource:0}: Error finding container 5e9bb39ce79f2a386f1a4bd1ed0f01ae29ebb7fe47d70330502d542886bb6914: Status 404 returned error can't find the container with id 5e9bb39ce79f2a386f1a4bd1ed0f01ae29ebb7fe47d70330502d542886bb6914
Oct 30 22:11:27.484913 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:11:27.484887    6133 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d4t4g\" (UniqueName: \"kubernetes.io/projected/fbd531c4-ba6f-4632-b6f2-d1fd9dead011-kube-api-access-d4t4g\") pod \"isvc-xgboost-graph-raw-hpa-a5a79-predictor-869d547bd8-f9hpm\" (UID: \"fbd531c4-ba6f-4632-b6f2-d1fd9dead011\") " pod="e2e-gw0-7b6759dd5d/isvc-xgboost-graph-raw-hpa-a5a79-predictor-869d547bd8-f9hpm"
Oct 30 22:11:27.556042 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:11:27.556002    6133 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-7b6759dd5d/isvc-xgboost-graph-raw-hpa-a5a79-predictor-869d547bd8-f9hpm"
Oct 30 22:11:27.865511 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:11:27.865484    6133 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw0-7b6759dd5d/isvc-xgboost-graph-raw-hpa-a5a79-predictor-869d547bd8-f9hpm"]
Oct 30 22:11:27.867361 ip-10-0-143-246 kubenswrapper[6133]: W1030 22:11:27.867330    6133 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfbd531c4_ba6f_4632_b6f2_d1fd9dead011.slice/crio-8b7af79ab650c3835ea2bbeff6075c1cee32c1da5b48dc4f2ae9c5ae9b494082 WatchSource:0}: Error finding container 8b7af79ab650c3835ea2bbeff6075c1cee32c1da5b48dc4f2ae9c5ae9b494082: Status 404 returned error can't find the container with id 8b7af79ab650c3835ea2bbeff6075c1cee32c1da5b48dc4f2ae9c5ae9b494082
Oct 30 22:11:28.227112 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:11:28.227014    6133 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-7b6759dd5d/isvc-xgboost-graph-raw-hpa-a5a79-predictor-869d547bd8-f9hpm" event={"ID":"fbd531c4-ba6f-4632-b6f2-d1fd9dead011","Type":"ContainerStarted","Data":"4faf5b4deba105fd01fa2e927cf2cfc0cea71b8af03101e97e3d1a56437eda20"}
Oct 30 22:11:28.227112 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:11:28.227085    6133 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-7b6759dd5d/isvc-xgboost-graph-raw-hpa-a5a79-predictor-869d547bd8-f9hpm" event={"ID":"fbd531c4-ba6f-4632-b6f2-d1fd9dead011","Type":"ContainerStarted","Data":"8b7af79ab650c3835ea2bbeff6075c1cee32c1da5b48dc4f2ae9c5ae9b494082"}
Oct 30 22:11:28.228688 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:11:28.228662    6133 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-7b6759dd5d/isvc-sklearn-graph-raw-hpa-a5a79-predictor-5f647f9b68-5t9lb" event={"ID":"b184588e-b671-4841-b1fe-ee453ee1116c","Type":"ContainerStarted","Data":"fdfa80581eb4f8584881b37641a96a2b395f7470ad9ed13a168a11ad9c86a138"}
Oct 30 22:11:28.228807 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:11:28.228709    6133 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-7b6759dd5d/isvc-sklearn-graph-raw-hpa-a5a79-predictor-5f647f9b68-5t9lb" event={"ID":"b184588e-b671-4841-b1fe-ee453ee1116c","Type":"ContainerStarted","Data":"5e9bb39ce79f2a386f1a4bd1ed0f01ae29ebb7fe47d70330502d542886bb6914"}
Oct 30 22:11:32.017526 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:11:32.017487    6133 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-2e5a75ca98/isvc-canary-stop-predictor-7f5d58797c-7dm5f" podUID="bb80ffe3-13a0-4765-aa4f-4e74eb4f0a42" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.23:8080: connect: connection refused"
Oct 30 22:11:32.247188 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:11:32.247153    6133 generic.go:356] "Generic (PLEG): container finished" podID="fbd531c4-ba6f-4632-b6f2-d1fd9dead011" containerID="4faf5b4deba105fd01fa2e927cf2cfc0cea71b8af03101e97e3d1a56437eda20" exitCode=0
Oct 30 22:11:32.247371 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:11:32.247227    6133 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-7b6759dd5d/isvc-xgboost-graph-raw-hpa-a5a79-predictor-869d547bd8-f9hpm" event={"ID":"fbd531c4-ba6f-4632-b6f2-d1fd9dead011","Type":"ContainerDied","Data":"4faf5b4deba105fd01fa2e927cf2cfc0cea71b8af03101e97e3d1a56437eda20"}
Oct 30 22:11:32.249025 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:11:32.249000    6133 generic.go:356] "Generic (PLEG): container finished" podID="b184588e-b671-4841-b1fe-ee453ee1116c" containerID="fdfa80581eb4f8584881b37641a96a2b395f7470ad9ed13a168a11ad9c86a138" exitCode=0
Oct 30 22:11:32.249136 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:11:32.249024    6133 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-7b6759dd5d/isvc-sklearn-graph-raw-hpa-a5a79-predictor-5f647f9b68-5t9lb" event={"ID":"b184588e-b671-4841-b1fe-ee453ee1116c","Type":"ContainerDied","Data":"fdfa80581eb4f8584881b37641a96a2b395f7470ad9ed13a168a11ad9c86a138"}
Oct 30 22:11:33.255907 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:11:33.255871    6133 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-7b6759dd5d/isvc-sklearn-graph-raw-hpa-a5a79-predictor-5f647f9b68-5t9lb" event={"ID":"b184588e-b671-4841-b1fe-ee453ee1116c","Type":"ContainerStarted","Data":"ac51d973832df8a8bd3e0353d200bace409c880698ac56ce5513bf2c087b1c1c"}
Oct 30 22:11:33.256355 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:11:33.255934    6133 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-7b6759dd5d/isvc-sklearn-graph-raw-hpa-a5a79-predictor-5f647f9b68-5t9lb" event={"ID":"b184588e-b671-4841-b1fe-ee453ee1116c","Type":"ContainerStarted","Data":"509632e56c5f7a8e7d6a37390635f4bc09f730865b32f6e1965a0169df74c3f2"}
Oct 30 22:11:33.256355 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:11:33.256235    6133 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw0-7b6759dd5d/isvc-sklearn-graph-raw-hpa-a5a79-predictor-5f647f9b68-5t9lb"
Oct 30 22:11:33.256500 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:11:33.256388    6133 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw0-7b6759dd5d/isvc-sklearn-graph-raw-hpa-a5a79-predictor-5f647f9b68-5t9lb"
Oct 30 22:11:33.257901 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:11:33.257877    6133 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7b6759dd5d/isvc-sklearn-graph-raw-hpa-a5a79-predictor-5f647f9b68-5t9lb" podUID="b184588e-b671-4841-b1fe-ee453ee1116c" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.24:8080: connect: connection refused"
Oct 30 22:11:33.258405 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:11:33.258385    6133 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-7b6759dd5d/isvc-xgboost-graph-raw-hpa-a5a79-predictor-869d547bd8-f9hpm" event={"ID":"fbd531c4-ba6f-4632-b6f2-d1fd9dead011","Type":"ContainerStarted","Data":"41fb903621fa24f1acec9d6c4a4c6d2d5a6b35b7913b66c6775730c60b5ef01c"}
Oct 30 22:11:33.258488 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:11:33.258428    6133 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-7b6759dd5d/isvc-xgboost-graph-raw-hpa-a5a79-predictor-869d547bd8-f9hpm" event={"ID":"fbd531c4-ba6f-4632-b6f2-d1fd9dead011","Type":"ContainerStarted","Data":"aa23e9cb5ee43bcdc937436b9169f81b96a410da34694d47456f895773218731"}
Oct 30 22:11:33.258607 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:11:33.258593    6133 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw0-7b6759dd5d/isvc-xgboost-graph-raw-hpa-a5a79-predictor-869d547bd8-f9hpm"
Oct 30 22:11:33.317486 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:11:33.317416    6133 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-gw0-7b6759dd5d/isvc-sklearn-graph-raw-hpa-a5a79-predictor-5f647f9b68-5t9lb" podStartSLOduration=3.463537344 podStartE2EDuration="7.317401803s" totalImagesPullingTime="0s" totalInitContainerRuntime="3.853864459s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:11:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:11:33.316178383 +0000 UTC m=+1000.076934294" watchObservedRunningTime="2026-10-30 22:11:33.317401803 +0000 UTC m=+1000.078157689"
Oct 30 22:11:33.371852 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:11:33.371787    6133 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-gw0-7b6759dd5d/isvc-xgboost-graph-raw-hpa-a5a79-predictor-869d547bd8-f9hpm" podStartSLOduration=2.762091772 podStartE2EDuration="6.371771072s" totalImagesPullingTime="0s" totalInitContainerRuntime="3.6096793s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:11:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:11:33.370178522 +0000 UTC m=+1000.130934411" watchObservedRunningTime="2026-10-30 22:11:33.371771072 +0000 UTC m=+1000.132526962"
Oct 30 22:11:34.262789 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:11:34.262743    6133 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7b6759dd5d/isvc-sklearn-graph-raw-hpa-a5a79-predictor-5f647f9b68-5t9lb" podUID="b184588e-b671-4841-b1fe-ee453ee1116c" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.24:8080: connect: connection refused"
Oct 30 22:11:34.263287 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:11:34.263022    6133 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw0-7b6759dd5d/isvc-xgboost-graph-raw-hpa-a5a79-predictor-869d547bd8-f9hpm"
Oct 30 22:11:34.264195 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:11:34.264160    6133 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7b6759dd5d/isvc-xgboost-graph-raw-hpa-a5a79-predictor-869d547bd8-f9hpm" podUID="fbd531c4-ba6f-4632-b6f2-d1fd9dead011" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.25:8080: connect: connection refused"
Oct 30 22:11:35.267210 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:11:35.267122    6133 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7b6759dd5d/isvc-xgboost-graph-raw-hpa-a5a79-predictor-869d547bd8-f9hpm" podUID="fbd531c4-ba6f-4632-b6f2-d1fd9dead011" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.25:8080: connect: connection refused"
Oct 30 22:11:39.267392 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:11:39.267361    6133 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw0-7b6759dd5d/isvc-sklearn-graph-raw-hpa-a5a79-predictor-5f647f9b68-5t9lb"
Oct 30 22:11:39.267977 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:11:39.267948    6133 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7b6759dd5d/isvc-sklearn-graph-raw-hpa-a5a79-predictor-5f647f9b68-5t9lb" podUID="b184588e-b671-4841-b1fe-ee453ee1116c" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.24:8080: connect: connection refused"
Oct 30 22:11:39.526740 ip-10-0-143-246 kubenswrapper[6133]: E1030 22:11:39.526660    6133 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6jppj" containerName="extractor"
Oct 30 22:11:40.272186 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:11:40.272156    6133 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw0-7b6759dd5d/isvc-xgboost-graph-raw-hpa-a5a79-predictor-869d547bd8-f9hpm"
Oct 30 22:11:40.272966 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:11:40.272813    6133 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7b6759dd5d/isvc-xgboost-graph-raw-hpa-a5a79-predictor-869d547bd8-f9hpm" podUID="fbd531c4-ba6f-4632-b6f2-d1fd9dead011" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.25:8080: connect: connection refused"
Oct 30 22:11:42.017010 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:11:42.016956    6133 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-2e5a75ca98/isvc-canary-stop-predictor-7f5d58797c-7dm5f" podUID="bb80ffe3-13a0-4765-aa4f-4e74eb4f0a42" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.23:8080: connect: connection refused"
Oct 30 22:11:49.268242 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:11:49.268195    6133 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7b6759dd5d/isvc-sklearn-graph-raw-hpa-a5a79-predictor-5f647f9b68-5t9lb" podUID="b184588e-b671-4841-b1fe-ee453ee1116c" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.24:8080: connect: connection refused"
Oct 30 22:11:50.272820 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:11:50.272776    6133 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7b6759dd5d/isvc-xgboost-graph-raw-hpa-a5a79-predictor-869d547bd8-f9hpm" podUID="fbd531c4-ba6f-4632-b6f2-d1fd9dead011" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.25:8080: connect: connection refused"
Oct 30 22:11:52.017248 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:11:52.017211    6133 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-2e5a75ca98/isvc-canary-stop-predictor-7f5d58797c-7dm5f" podUID="bb80ffe3-13a0-4765-aa4f-4e74eb4f0a42" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.23:8080: connect: connection refused"
Oct 30 22:11:54.351165 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:11:54.351132    6133 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw1-2e5a75ca98/isvc-canary-stop-predictor-7f5d58797c-7dm5f"
Oct 30 22:11:59.268342 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:11:59.268301    6133 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7b6759dd5d/isvc-sklearn-graph-raw-hpa-a5a79-predictor-5f647f9b68-5t9lb" podUID="b184588e-b671-4841-b1fe-ee453ee1116c" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.24:8080: connect: connection refused"
Oct 30 22:12:00.272938 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:12:00.272894    6133 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7b6759dd5d/isvc-xgboost-graph-raw-hpa-a5a79-predictor-869d547bd8-f9hpm" podUID="fbd531c4-ba6f-4632-b6f2-d1fd9dead011" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.25:8080: connect: connection refused"
Oct 30 22:12:04.191192 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:12:04.191157    6133 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw1-2e5a75ca98/isvc-canary-stop-predictor-7f5d58797c-7dm5f"]
Oct 30 22:12:04.191687 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:12:04.191390    6133 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw1-2e5a75ca98/isvc-canary-stop-predictor-7f5d58797c-7dm5f" podUID="bb80ffe3-13a0-4765-aa4f-4e74eb4f0a42" containerName="kserve-container" containerID="cri-o://3e5ed522261609c1606663c2c8ebfc918a4ec347764b87644bb40025e89c0998" gracePeriod=30
Oct 30 22:12:04.191687 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:12:04.191439    6133 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw1-2e5a75ca98/isvc-canary-stop-predictor-7f5d58797c-7dm5f" podUID="bb80ffe3-13a0-4765-aa4f-4e74eb4f0a42" containerName="kube-rbac-proxy" containerID="cri-o://37d57c215a0b1d6a319a925596761d12caebd37558acfc444853ed17cdaee6fc" gracePeriod=30
Oct 30 22:12:04.350687 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:12:04.350647    6133 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-2e5a75ca98/isvc-canary-stop-predictor-7f5d58797c-7dm5f" podUID="bb80ffe3-13a0-4765-aa4f-4e74eb4f0a42" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.23:8080: connect: connection refused"
Oct 30 22:12:04.385383 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:12:04.385352    6133 generic.go:356] "Generic (PLEG): container finished" podID="bb80ffe3-13a0-4765-aa4f-4e74eb4f0a42" containerID="37d57c215a0b1d6a319a925596761d12caebd37558acfc444853ed17cdaee6fc" exitCode=0
Oct 30 22:12:04.385556 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:12:04.385421    6133 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-2e5a75ca98/isvc-canary-stop-predictor-7f5d58797c-7dm5f" event={"ID":"bb80ffe3-13a0-4765-aa4f-4e74eb4f0a42","Type":"ContainerDied","Data":"37d57c215a0b1d6a319a925596761d12caebd37558acfc444853ed17cdaee6fc"}
Oct 30 22:12:07.012249 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:12:07.012202    6133 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-2e5a75ca98/isvc-canary-stop-predictor-7f5d58797c-7dm5f" podUID="bb80ffe3-13a0-4765-aa4f-4e74eb4f0a42" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.23:8643/healthz\": dial tcp 10.134.0.23:8643: connect: connection refused"
Oct 30 22:12:09.268244 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:12:09.268201    6133 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7b6759dd5d/isvc-sklearn-graph-raw-hpa-a5a79-predictor-5f647f9b68-5t9lb" podUID="b184588e-b671-4841-b1fe-ee453ee1116c" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.24:8080: connect: connection refused"
Oct 30 22:12:09.527614 ip-10-0-143-246 kubenswrapper[6133]: E1030 22:12:09.527526    6133 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6jppj" containerName="extractor"
Oct 30 22:12:09.552919 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:12:09.552889    6133 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-2e5a75ca98/isvc-canary-stop-predictor-7f5d58797c-7dm5f"
Oct 30 22:12:09.613159 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:12:09.613121    6133 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/bb80ffe3-13a0-4765-aa4f-4e74eb4f0a42-kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/bb80ffe3-13a0-4765-aa4f-4e74eb4f0a42-kube-rbac-proxy-sar-config\") pod \"bb80ffe3-13a0-4765-aa4f-4e74eb4f0a42\" (UID: \"bb80ffe3-13a0-4765-aa4f-4e74eb4f0a42\") "
Oct 30 22:12:09.613349 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:12:09.613178    6133 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/bb80ffe3-13a0-4765-aa4f-4e74eb4f0a42-cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/bb80ffe3-13a0-4765-aa4f-4e74eb4f0a42-cabundle-cert\") pod \"bb80ffe3-13a0-4765-aa4f-4e74eb4f0a42\" (UID: \"bb80ffe3-13a0-4765-aa4f-4e74eb4f0a42\") "
Oct 30 22:12:09.613349 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:12:09.613228    6133 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bb80ffe3-13a0-4765-aa4f-4e74eb4f0a42-kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/bb80ffe3-13a0-4765-aa4f-4e74eb4f0a42-kserve-provision-location\") pod \"bb80ffe3-13a0-4765-aa4f-4e74eb4f0a42\" (UID: \"bb80ffe3-13a0-4765-aa4f-4e74eb4f0a42\") "
Oct 30 22:12:09.613349 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:12:09.613261    6133 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/bb80ffe3-13a0-4765-aa4f-4e74eb4f0a42-kube-api-access-6t5h5\" (UniqueName: \"kubernetes.io/projected/bb80ffe3-13a0-4765-aa4f-4e74eb4f0a42-kube-api-access-6t5h5\") pod \"bb80ffe3-13a0-4765-aa4f-4e74eb4f0a42\" (UID: \"bb80ffe3-13a0-4765-aa4f-4e74eb4f0a42\") "
Oct 30 22:12:09.613349 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:12:09.613296    6133 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/bb80ffe3-13a0-4765-aa4f-4e74eb4f0a42-proxy-tls\" (UniqueName: \"kubernetes.io/secret/bb80ffe3-13a0-4765-aa4f-4e74eb4f0a42-proxy-tls\") pod \"bb80ffe3-13a0-4765-aa4f-4e74eb4f0a42\" (UID: \"bb80ffe3-13a0-4765-aa4f-4e74eb4f0a42\") "
Oct 30 22:12:09.613584 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:12:09.613555    6133 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bb80ffe3-13a0-4765-aa4f-4e74eb4f0a42-kserve-provision-location" pod "bb80ffe3-13a0-4765-aa4f-4e74eb4f0a42" (UID: "bb80ffe3-13a0-4765-aa4f-4e74eb4f0a42"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:12:09.613637 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:12:09.613581    6133 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bb80ffe3-13a0-4765-aa4f-4e74eb4f0a42-kube-rbac-proxy-sar-config" pod "bb80ffe3-13a0-4765-aa4f-4e74eb4f0a42" (UID: "bb80ffe3-13a0-4765-aa4f-4e74eb4f0a42"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:12:09.613637 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:12:09.613586    6133 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bb80ffe3-13a0-4765-aa4f-4e74eb4f0a42-cabundle-cert" pod "bb80ffe3-13a0-4765-aa4f-4e74eb4f0a42" (UID: "bb80ffe3-13a0-4765-aa4f-4e74eb4f0a42"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:12:09.615702 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:12:09.615670    6133 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bb80ffe3-13a0-4765-aa4f-4e74eb4f0a42-proxy-tls" pod "bb80ffe3-13a0-4765-aa4f-4e74eb4f0a42" (UID: "bb80ffe3-13a0-4765-aa4f-4e74eb4f0a42"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:12:09.615824 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:12:09.615700    6133 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bb80ffe3-13a0-4765-aa4f-4e74eb4f0a42-kube-api-access-6t5h5" pod "bb80ffe3-13a0-4765-aa4f-4e74eb4f0a42" (UID: "bb80ffe3-13a0-4765-aa4f-4e74eb4f0a42"). InnerVolumeSpecName "kube-api-access-6t5h5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:12:09.714330 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:12:09.714270    6133 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/bb80ffe3-13a0-4765-aa4f-4e74eb4f0a42-kube-rbac-proxy-sar-config\") on node \"ip-10-0-143-246.ec2.internal\" DevicePath \"\""
Oct 30 22:12:09.714330 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:12:09.714321    6133 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/bb80ffe3-13a0-4765-aa4f-4e74eb4f0a42-cabundle-cert\") on node \"ip-10-0-143-246.ec2.internal\" DevicePath \"\""
Oct 30 22:12:09.714330 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:12:09.714336    6133 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/bb80ffe3-13a0-4765-aa4f-4e74eb4f0a42-kserve-provision-location\") on node \"ip-10-0-143-246.ec2.internal\" DevicePath \"\""
Oct 30 22:12:09.714330 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:12:09.714349    6133 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6t5h5\" (UniqueName: \"kubernetes.io/projected/bb80ffe3-13a0-4765-aa4f-4e74eb4f0a42-kube-api-access-6t5h5\") on node \"ip-10-0-143-246.ec2.internal\" DevicePath \"\""
Oct 30 22:12:09.714611 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:12:09.714363    6133 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/bb80ffe3-13a0-4765-aa4f-4e74eb4f0a42-proxy-tls\") on node \"ip-10-0-143-246.ec2.internal\" DevicePath \"\""
Oct 30 22:12:10.273700 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:12:10.273662    6133 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7b6759dd5d/isvc-xgboost-graph-raw-hpa-a5a79-predictor-869d547bd8-f9hpm" podUID="fbd531c4-ba6f-4632-b6f2-d1fd9dead011" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.25:8080: connect: connection refused"
Oct 30 22:12:10.409303 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:12:10.409269    6133 generic.go:356] "Generic (PLEG): container finished" podID="bb80ffe3-13a0-4765-aa4f-4e74eb4f0a42" containerID="3e5ed522261609c1606663c2c8ebfc918a4ec347764b87644bb40025e89c0998" exitCode=0
Oct 30 22:12:10.409468 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:12:10.409340    6133 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-2e5a75ca98/isvc-canary-stop-predictor-7f5d58797c-7dm5f"
Oct 30 22:12:10.409468 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:12:10.409350    6133 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-2e5a75ca98/isvc-canary-stop-predictor-7f5d58797c-7dm5f" event={"ID":"bb80ffe3-13a0-4765-aa4f-4e74eb4f0a42","Type":"ContainerDied","Data":"3e5ed522261609c1606663c2c8ebfc918a4ec347764b87644bb40025e89c0998"}
Oct 30 22:12:10.409468 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:12:10.409415    6133 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-2e5a75ca98/isvc-canary-stop-predictor-7f5d58797c-7dm5f" event={"ID":"bb80ffe3-13a0-4765-aa4f-4e74eb4f0a42","Type":"ContainerDied","Data":"af8c2273bf0f73aae26f430681db5883908c40fef58b952583d51b6f301a6a42"}
Oct 30 22:12:10.409468 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:12:10.409442    6133 scope.go:122] "RemoveContainer" containerID="37d57c215a0b1d6a319a925596761d12caebd37558acfc444853ed17cdaee6fc"
Oct 30 22:12:10.418203 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:12:10.418068    6133 scope.go:122] "RemoveContainer" containerID="3e5ed522261609c1606663c2c8ebfc918a4ec347764b87644bb40025e89c0998"
Oct 30 22:12:10.425621 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:12:10.425601    6133 scope.go:122] "RemoveContainer" containerID="2faa92a3f45a5d492d9b8fab8c2d6284f8eb997e39b7d8374c016ea57b3736f0"
Oct 30 22:12:10.433158 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:12:10.433131    6133 scope.go:122] "RemoveContainer" containerID="37d57c215a0b1d6a319a925596761d12caebd37558acfc444853ed17cdaee6fc"
Oct 30 22:12:10.433453 ip-10-0-143-246 kubenswrapper[6133]: E1030 22:12:10.433424    6133 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"37d57c215a0b1d6a319a925596761d12caebd37558acfc444853ed17cdaee6fc\": container with ID starting with 37d57c215a0b1d6a319a925596761d12caebd37558acfc444853ed17cdaee6fc not found: ID does not exist" containerID="37d57c215a0b1d6a319a925596761d12caebd37558acfc444853ed17cdaee6fc"
Oct 30 22:12:10.433554 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:12:10.433461    6133 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"37d57c215a0b1d6a319a925596761d12caebd37558acfc444853ed17cdaee6fc"} err="failed to get container status \"37d57c215a0b1d6a319a925596761d12caebd37558acfc444853ed17cdaee6fc\": rpc error: code = NotFound desc = could not find container \"37d57c215a0b1d6a319a925596761d12caebd37558acfc444853ed17cdaee6fc\": container with ID starting with 37d57c215a0b1d6a319a925596761d12caebd37558acfc444853ed17cdaee6fc not found: ID does not exist"
Oct 30 22:12:10.433554 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:12:10.433481    6133 scope.go:122] "RemoveContainer" containerID="3e5ed522261609c1606663c2c8ebfc918a4ec347764b87644bb40025e89c0998"
Oct 30 22:12:10.433764 ip-10-0-143-246 kubenswrapper[6133]: E1030 22:12:10.433749    6133 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3e5ed522261609c1606663c2c8ebfc918a4ec347764b87644bb40025e89c0998\": container with ID starting with 3e5ed522261609c1606663c2c8ebfc918a4ec347764b87644bb40025e89c0998 not found: ID does not exist" containerID="3e5ed522261609c1606663c2c8ebfc918a4ec347764b87644bb40025e89c0998"
Oct 30 22:12:10.433814 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:12:10.433767    6133 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3e5ed522261609c1606663c2c8ebfc918a4ec347764b87644bb40025e89c0998"} err="failed to get container status \"3e5ed522261609c1606663c2c8ebfc918a4ec347764b87644bb40025e89c0998\": rpc error: code = NotFound desc = could not find container \"3e5ed522261609c1606663c2c8ebfc918a4ec347764b87644bb40025e89c0998\": container with ID starting with 3e5ed522261609c1606663c2c8ebfc918a4ec347764b87644bb40025e89c0998 not found: ID does not exist"
Oct 30 22:12:10.433814 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:12:10.433779    6133 scope.go:122] "RemoveContainer" containerID="2faa92a3f45a5d492d9b8fab8c2d6284f8eb997e39b7d8374c016ea57b3736f0"
Oct 30 22:12:10.433977 ip-10-0-143-246 kubenswrapper[6133]: E1030 22:12:10.433958    6133 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2faa92a3f45a5d492d9b8fab8c2d6284f8eb997e39b7d8374c016ea57b3736f0\": container with ID starting with 2faa92a3f45a5d492d9b8fab8c2d6284f8eb997e39b7d8374c016ea57b3736f0 not found: ID does not exist" containerID="2faa92a3f45a5d492d9b8fab8c2d6284f8eb997e39b7d8374c016ea57b3736f0"
Oct 30 22:12:10.434327 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:12:10.433981    6133 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2faa92a3f45a5d492d9b8fab8c2d6284f8eb997e39b7d8374c016ea57b3736f0"} err="failed to get container status \"2faa92a3f45a5d492d9b8fab8c2d6284f8eb997e39b7d8374c016ea57b3736f0\": rpc error: code = NotFound desc = could not find container \"2faa92a3f45a5d492d9b8fab8c2d6284f8eb997e39b7d8374c016ea57b3736f0\": container with ID starting with 2faa92a3f45a5d492d9b8fab8c2d6284f8eb997e39b7d8374c016ea57b3736f0 not found: ID does not exist"
Oct 30 22:12:10.476819 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:12:10.476776    6133 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw1-2e5a75ca98/isvc-canary-stop-predictor-7f5d58797c-7dm5f"]
Oct 30 22:12:10.489015 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:12:10.488972    6133 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-gw1-2e5a75ca98/isvc-canary-stop-predictor-7f5d58797c-7dm5f"]
Oct 30 22:12:11.355191 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:12:11.355159    6133 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="bb80ffe3-13a0-4765-aa4f-4e74eb4f0a42" path="/var/lib/kubelet/pods/bb80ffe3-13a0-4765-aa4f-4e74eb4f0a42/volumes"
Oct 30 22:12:19.268899 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:12:19.268854    6133 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7b6759dd5d/isvc-sklearn-graph-raw-hpa-a5a79-predictor-5f647f9b68-5t9lb" podUID="b184588e-b671-4841-b1fe-ee453ee1116c" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.24:8080: connect: connection refused"
Oct 30 22:12:20.273179 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:12:20.273130    6133 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7b6759dd5d/isvc-xgboost-graph-raw-hpa-a5a79-predictor-869d547bd8-f9hpm" podUID="fbd531c4-ba6f-4632-b6f2-d1fd9dead011" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.25:8080: connect: connection refused"
Oct 30 22:12:29.268925 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:12:29.268877    6133 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7b6759dd5d/isvc-sklearn-graph-raw-hpa-a5a79-predictor-5f647f9b68-5t9lb" podUID="b184588e-b671-4841-b1fe-ee453ee1116c" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.24:8080: connect: connection refused"
Oct 30 22:12:30.272772 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:12:30.272723    6133 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7b6759dd5d/isvc-xgboost-graph-raw-hpa-a5a79-predictor-869d547bd8-f9hpm" podUID="fbd531c4-ba6f-4632-b6f2-d1fd9dead011" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.25:8080: connect: connection refused"
Oct 30 22:12:39.267890 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:12:39.267851    6133 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7b6759dd5d/isvc-sklearn-graph-raw-hpa-a5a79-predictor-5f647f9b68-5t9lb" podUID="b184588e-b671-4841-b1fe-ee453ee1116c" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.24:8080: connect: connection refused"
Oct 30 22:12:39.528456 ip-10-0-143-246 kubenswrapper[6133]: E1030 22:12:39.528371    6133 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6jppj" containerName="extractor"
Oct 30 22:12:40.272843 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:12:40.272799    6133 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7b6759dd5d/isvc-xgboost-graph-raw-hpa-a5a79-predictor-869d547bd8-f9hpm" podUID="fbd531c4-ba6f-4632-b6f2-d1fd9dead011" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.25:8080: connect: connection refused"
Oct 30 22:12:40.350942 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:12:40.350895    6133 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7b6759dd5d/isvc-sklearn-graph-raw-hpa-a5a79-predictor-5f647f9b68-5t9lb" podUID="b184588e-b671-4841-b1fe-ee453ee1116c" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.24:8080: connect: connection refused"
Oct 30 22:12:50.273889 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:12:50.273854    6133 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw0-7b6759dd5d/isvc-xgboost-graph-raw-hpa-a5a79-predictor-869d547bd8-f9hpm"
Oct 30 22:12:50.351825 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:12:50.351791    6133 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7b6759dd5d/isvc-sklearn-graph-raw-hpa-a5a79-predictor-5f647f9b68-5t9lb" podUID="b184588e-b671-4841-b1fe-ee453ee1116c" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.24:8080: connect: connection refused"
Oct 30 22:12:53.449496 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:12:53.449466    6133 scope.go:122] "RemoveContainer" containerID="15e513e01fe357c2ad2da4c26c43cf92e9a53795467a9a1647ad301c9f968e3a"
Oct 30 22:13:00.351396 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:13:00.351364    6133 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw0-7b6759dd5d/isvc-sklearn-graph-raw-hpa-a5a79-predictor-5f647f9b68-5t9lb"
Oct 30 22:13:09.527767 ip-10-0-143-246 kubenswrapper[6133]: E1030 22:13:09.527677    6133 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6jppj" containerName="extractor"
Oct 30 22:13:27.434049 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:13:27.434010    6133 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw0-7b6759dd5d/isvc-sklearn-graph-raw-hpa-a5a79-predictor-5f647f9b68-5t9lb"]
Oct 30 22:13:27.434561 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:13:27.434320    6133 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw0-7b6759dd5d/isvc-sklearn-graph-raw-hpa-a5a79-predictor-5f647f9b68-5t9lb" podUID="b184588e-b671-4841-b1fe-ee453ee1116c" containerName="kserve-container" containerID="cri-o://509632e56c5f7a8e7d6a37390635f4bc09f730865b32f6e1965a0169df74c3f2" gracePeriod=30
Oct 30 22:13:27.434561 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:13:27.434345    6133 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw0-7b6759dd5d/isvc-sklearn-graph-raw-hpa-a5a79-predictor-5f647f9b68-5t9lb" podUID="b184588e-b671-4841-b1fe-ee453ee1116c" containerName="kube-rbac-proxy" containerID="cri-o://ac51d973832df8a8bd3e0353d200bace409c880698ac56ce5513bf2c087b1c1c" gracePeriod=30
Oct 30 22:13:27.632545 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:13:27.632503    6133 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw0-7b6759dd5d/isvc-xgboost-graph-raw-hpa-a5a79-predictor-869d547bd8-f9hpm"]
Oct 30 22:13:27.632848 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:13:27.632821    6133 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw0-7b6759dd5d/isvc-xgboost-graph-raw-hpa-a5a79-predictor-869d547bd8-f9hpm" podUID="fbd531c4-ba6f-4632-b6f2-d1fd9dead011" containerName="kserve-container" containerID="cri-o://aa23e9cb5ee43bcdc937436b9169f81b96a410da34694d47456f895773218731" gracePeriod=30
Oct 30 22:13:27.632940 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:13:27.632867    6133 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw0-7b6759dd5d/isvc-xgboost-graph-raw-hpa-a5a79-predictor-869d547bd8-f9hpm" podUID="fbd531c4-ba6f-4632-b6f2-d1fd9dead011" containerName="kube-rbac-proxy" containerID="cri-o://41fb903621fa24f1acec9d6c4a4c6d2d5a6b35b7913b66c6775730c60b5ef01c" gracePeriod=30
Oct 30 22:13:27.702600 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:13:27.702513    6133 generic.go:356] "Generic (PLEG): container finished" podID="b184588e-b671-4841-b1fe-ee453ee1116c" containerID="ac51d973832df8a8bd3e0353d200bace409c880698ac56ce5513bf2c087b1c1c" exitCode=0
Oct 30 22:13:27.702600 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:13:27.702585    6133 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-7b6759dd5d/isvc-sklearn-graph-raw-hpa-a5a79-predictor-5f647f9b68-5t9lb" event={"ID":"b184588e-b671-4841-b1fe-ee453ee1116c","Type":"ContainerDied","Data":"ac51d973832df8a8bd3e0353d200bace409c880698ac56ce5513bf2c087b1c1c"}
Oct 30 22:13:28.708718 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:13:28.708633    6133 generic.go:356] "Generic (PLEG): container finished" podID="fbd531c4-ba6f-4632-b6f2-d1fd9dead011" containerID="41fb903621fa24f1acec9d6c4a4c6d2d5a6b35b7913b66c6775730c60b5ef01c" exitCode=0
Oct 30 22:13:28.708718 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:13:28.708697    6133 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-7b6759dd5d/isvc-xgboost-graph-raw-hpa-a5a79-predictor-869d547bd8-f9hpm" event={"ID":"fbd531c4-ba6f-4632-b6f2-d1fd9dead011","Type":"ContainerDied","Data":"41fb903621fa24f1acec9d6c4a4c6d2d5a6b35b7913b66c6775730c60b5ef01c"}
Oct 30 22:13:29.262954 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:13:29.262913    6133 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7b6759dd5d/isvc-sklearn-graph-raw-hpa-a5a79-predictor-5f647f9b68-5t9lb" podUID="b184588e-b671-4841-b1fe-ee453ee1116c" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.24:8643/healthz\": dial tcp 10.134.0.24:8643: connect: connection refused"
Oct 30 22:13:30.268337 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:13:30.268299    6133 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7b6759dd5d/isvc-xgboost-graph-raw-hpa-a5a79-predictor-869d547bd8-f9hpm" podUID="fbd531c4-ba6f-4632-b6f2-d1fd9dead011" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.25:8643/healthz\": dial tcp 10.134.0.25:8643: connect: connection refused"
Oct 30 22:13:30.272794 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:13:30.272750    6133 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7b6759dd5d/isvc-xgboost-graph-raw-hpa-a5a79-predictor-869d547bd8-f9hpm" podUID="fbd531c4-ba6f-4632-b6f2-d1fd9dead011" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.25:8080: connect: connection refused"
Oct 30 22:13:30.351207 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:13:30.351167    6133 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7b6759dd5d/isvc-sklearn-graph-raw-hpa-a5a79-predictor-5f647f9b68-5t9lb" podUID="b184588e-b671-4841-b1fe-ee453ee1116c" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.24:8080: connect: connection refused"
Oct 30 22:13:31.987001 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:13:31.986963    6133 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-7b6759dd5d/isvc-xgboost-graph-raw-hpa-a5a79-predictor-869d547bd8-f9hpm"
Oct 30 22:13:32.106271 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:13:32.106178    6133 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fbd531c4-ba6f-4632-b6f2-d1fd9dead011-kube-api-access-d4t4g\" (UniqueName: \"kubernetes.io/projected/fbd531c4-ba6f-4632-b6f2-d1fd9dead011-kube-api-access-d4t4g\") pod \"fbd531c4-ba6f-4632-b6f2-d1fd9dead011\" (UID: \"fbd531c4-ba6f-4632-b6f2-d1fd9dead011\") "
Oct 30 22:13:32.106271 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:13:32.106226    6133 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/fbd531c4-ba6f-4632-b6f2-d1fd9dead011-proxy-tls\" (UniqueName: \"kubernetes.io/secret/fbd531c4-ba6f-4632-b6f2-d1fd9dead011-proxy-tls\") pod \"fbd531c4-ba6f-4632-b6f2-d1fd9dead011\" (UID: \"fbd531c4-ba6f-4632-b6f2-d1fd9dead011\") "
Oct 30 22:13:32.106517 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:13:32.106273    6133 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fbd531c4-ba6f-4632-b6f2-d1fd9dead011-kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/fbd531c4-ba6f-4632-b6f2-d1fd9dead011-kserve-provision-location\") pod \"fbd531c4-ba6f-4632-b6f2-d1fd9dead011\" (UID: \"fbd531c4-ba6f-4632-b6f2-d1fd9dead011\") "
Oct 30 22:13:32.106517 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:13:32.106305    6133 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/fbd531c4-ba6f-4632-b6f2-d1fd9dead011-cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/fbd531c4-ba6f-4632-b6f2-d1fd9dead011-cabundle-cert\") pod \"fbd531c4-ba6f-4632-b6f2-d1fd9dead011\" (UID: \"fbd531c4-ba6f-4632-b6f2-d1fd9dead011\") "
Oct 30 22:13:32.106517 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:13:32.106326    6133 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/fbd531c4-ba6f-4632-b6f2-d1fd9dead011-kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/fbd531c4-ba6f-4632-b6f2-d1fd9dead011-kube-rbac-proxy-sar-config\") pod \"fbd531c4-ba6f-4632-b6f2-d1fd9dead011\" (UID: \"fbd531c4-ba6f-4632-b6f2-d1fd9dead011\") "
Oct 30 22:13:32.106685 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:13:32.106657    6133 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fbd531c4-ba6f-4632-b6f2-d1fd9dead011-kserve-provision-location" pod "fbd531c4-ba6f-4632-b6f2-d1fd9dead011" (UID: "fbd531c4-ba6f-4632-b6f2-d1fd9dead011"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:13:32.106736 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:13:32.106712    6133 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fbd531c4-ba6f-4632-b6f2-d1fd9dead011-cabundle-cert" pod "fbd531c4-ba6f-4632-b6f2-d1fd9dead011" (UID: "fbd531c4-ba6f-4632-b6f2-d1fd9dead011"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:13:32.106787 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:13:32.106725    6133 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fbd531c4-ba6f-4632-b6f2-d1fd9dead011-kube-rbac-proxy-sar-config" pod "fbd531c4-ba6f-4632-b6f2-d1fd9dead011" (UID: "fbd531c4-ba6f-4632-b6f2-d1fd9dead011"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:13:32.108847 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:13:32.108822    6133 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fbd531c4-ba6f-4632-b6f2-d1fd9dead011-proxy-tls" pod "fbd531c4-ba6f-4632-b6f2-d1fd9dead011" (UID: "fbd531c4-ba6f-4632-b6f2-d1fd9dead011"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:13:32.108847 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:13:32.108823    6133 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fbd531c4-ba6f-4632-b6f2-d1fd9dead011-kube-api-access-d4t4g" pod "fbd531c4-ba6f-4632-b6f2-d1fd9dead011" (UID: "fbd531c4-ba6f-4632-b6f2-d1fd9dead011"). InnerVolumeSpecName "kube-api-access-d4t4g". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:13:32.207198 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:13:32.207162    6133 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d4t4g\" (UniqueName: \"kubernetes.io/projected/fbd531c4-ba6f-4632-b6f2-d1fd9dead011-kube-api-access-d4t4g\") on node \"ip-10-0-143-246.ec2.internal\" DevicePath \"\""
Oct 30 22:13:32.207198 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:13:32.207192    6133 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/fbd531c4-ba6f-4632-b6f2-d1fd9dead011-proxy-tls\") on node \"ip-10-0-143-246.ec2.internal\" DevicePath \"\""
Oct 30 22:13:32.207198 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:13:32.207201    6133 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/fbd531c4-ba6f-4632-b6f2-d1fd9dead011-kserve-provision-location\") on node \"ip-10-0-143-246.ec2.internal\" DevicePath \"\""
Oct 30 22:13:32.207431 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:13:32.207209    6133 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/fbd531c4-ba6f-4632-b6f2-d1fd9dead011-cabundle-cert\") on node \"ip-10-0-143-246.ec2.internal\" DevicePath \"\""
Oct 30 22:13:32.207431 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:13:32.207219    6133 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/fbd531c4-ba6f-4632-b6f2-d1fd9dead011-kube-rbac-proxy-sar-config\") on node \"ip-10-0-143-246.ec2.internal\" DevicePath \"\""
Oct 30 22:13:32.725325 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:13:32.725294    6133 generic.go:356] "Generic (PLEG): container finished" podID="fbd531c4-ba6f-4632-b6f2-d1fd9dead011" containerID="aa23e9cb5ee43bcdc937436b9169f81b96a410da34694d47456f895773218731" exitCode=0
Oct 30 22:13:32.725486 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:13:32.725340    6133 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-7b6759dd5d/isvc-xgboost-graph-raw-hpa-a5a79-predictor-869d547bd8-f9hpm" event={"ID":"fbd531c4-ba6f-4632-b6f2-d1fd9dead011","Type":"ContainerDied","Data":"aa23e9cb5ee43bcdc937436b9169f81b96a410da34694d47456f895773218731"}
Oct 30 22:13:32.725486 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:13:32.725357    6133 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-7b6759dd5d/isvc-xgboost-graph-raw-hpa-a5a79-predictor-869d547bd8-f9hpm"
Oct 30 22:13:32.725486 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:13:32.725424    6133 scope.go:122] "RemoveContainer" containerID="41fb903621fa24f1acec9d6c4a4c6d2d5a6b35b7913b66c6775730c60b5ef01c"
Oct 30 22:13:32.725486 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:13:32.725413    6133 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-7b6759dd5d/isvc-xgboost-graph-raw-hpa-a5a79-predictor-869d547bd8-f9hpm" event={"ID":"fbd531c4-ba6f-4632-b6f2-d1fd9dead011","Type":"ContainerDied","Data":"8b7af79ab650c3835ea2bbeff6075c1cee32c1da5b48dc4f2ae9c5ae9b494082"}
Oct 30 22:13:32.728141 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:13:32.728119    6133 generic.go:356] "Generic (PLEG): container finished" podID="b184588e-b671-4841-b1fe-ee453ee1116c" containerID="509632e56c5f7a8e7d6a37390635f4bc09f730865b32f6e1965a0169df74c3f2" exitCode=0
Oct 30 22:13:32.728257 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:13:32.728174    6133 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-7b6759dd5d/isvc-sklearn-graph-raw-hpa-a5a79-predictor-5f647f9b68-5t9lb" event={"ID":"b184588e-b671-4841-b1fe-ee453ee1116c","Type":"ContainerDied","Data":"509632e56c5f7a8e7d6a37390635f4bc09f730865b32f6e1965a0169df74c3f2"}
Oct 30 22:13:32.740170 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:13:32.738182    6133 scope.go:122] "RemoveContainer" containerID="aa23e9cb5ee43bcdc937436b9169f81b96a410da34694d47456f895773218731"
Oct 30 22:13:32.746767 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:13:32.746744    6133 scope.go:122] "RemoveContainer" containerID="4faf5b4deba105fd01fa2e927cf2cfc0cea71b8af03101e97e3d1a56437eda20"
Oct 30 22:13:32.753932 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:13:32.753909    6133 scope.go:122] "RemoveContainer" containerID="41fb903621fa24f1acec9d6c4a4c6d2d5a6b35b7913b66c6775730c60b5ef01c"
Oct 30 22:13:32.754288 ip-10-0-143-246 kubenswrapper[6133]: E1030 22:13:32.754261    6133 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"41fb903621fa24f1acec9d6c4a4c6d2d5a6b35b7913b66c6775730c60b5ef01c\": container with ID starting with 41fb903621fa24f1acec9d6c4a4c6d2d5a6b35b7913b66c6775730c60b5ef01c not found: ID does not exist" containerID="41fb903621fa24f1acec9d6c4a4c6d2d5a6b35b7913b66c6775730c60b5ef01c"
Oct 30 22:13:32.754346 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:13:32.754301    6133 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"41fb903621fa24f1acec9d6c4a4c6d2d5a6b35b7913b66c6775730c60b5ef01c"} err="failed to get container status \"41fb903621fa24f1acec9d6c4a4c6d2d5a6b35b7913b66c6775730c60b5ef01c\": rpc error: code = NotFound desc = could not find container \"41fb903621fa24f1acec9d6c4a4c6d2d5a6b35b7913b66c6775730c60b5ef01c\": container with ID starting with 41fb903621fa24f1acec9d6c4a4c6d2d5a6b35b7913b66c6775730c60b5ef01c not found: ID does not exist"
Oct 30 22:13:32.754346 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:13:32.754326    6133 scope.go:122] "RemoveContainer" containerID="aa23e9cb5ee43bcdc937436b9169f81b96a410da34694d47456f895773218731"
Oct 30 22:13:32.754643 ip-10-0-143-246 kubenswrapper[6133]: E1030 22:13:32.754616    6133 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"aa23e9cb5ee43bcdc937436b9169f81b96a410da34694d47456f895773218731\": container with ID starting with aa23e9cb5ee43bcdc937436b9169f81b96a410da34694d47456f895773218731 not found: ID does not exist" containerID="aa23e9cb5ee43bcdc937436b9169f81b96a410da34694d47456f895773218731"
Oct 30 22:13:32.754741 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:13:32.754647    6133 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aa23e9cb5ee43bcdc937436b9169f81b96a410da34694d47456f895773218731"} err="failed to get container status \"aa23e9cb5ee43bcdc937436b9169f81b96a410da34694d47456f895773218731\": rpc error: code = NotFound desc = could not find container \"aa23e9cb5ee43bcdc937436b9169f81b96a410da34694d47456f895773218731\": container with ID starting with aa23e9cb5ee43bcdc937436b9169f81b96a410da34694d47456f895773218731 not found: ID does not exist"
Oct 30 22:13:32.754741 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:13:32.754663    6133 scope.go:122] "RemoveContainer" containerID="4faf5b4deba105fd01fa2e927cf2cfc0cea71b8af03101e97e3d1a56437eda20"
Oct 30 22:13:32.754912 ip-10-0-143-246 kubenswrapper[6133]: E1030 22:13:32.754895    6133 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4faf5b4deba105fd01fa2e927cf2cfc0cea71b8af03101e97e3d1a56437eda20\": container with ID starting with 4faf5b4deba105fd01fa2e927cf2cfc0cea71b8af03101e97e3d1a56437eda20 not found: ID does not exist" containerID="4faf5b4deba105fd01fa2e927cf2cfc0cea71b8af03101e97e3d1a56437eda20"
Oct 30 22:13:32.754953 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:13:32.754918    6133 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4faf5b4deba105fd01fa2e927cf2cfc0cea71b8af03101e97e3d1a56437eda20"} err="failed to get container status \"4faf5b4deba105fd01fa2e927cf2cfc0cea71b8af03101e97e3d1a56437eda20\": rpc error: code = NotFound desc = could not find container \"4faf5b4deba105fd01fa2e927cf2cfc0cea71b8af03101e97e3d1a56437eda20\": container with ID starting with 4faf5b4deba105fd01fa2e927cf2cfc0cea71b8af03101e97e3d1a56437eda20 not found: ID does not exist"
Oct 30 22:13:32.771495 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:13:32.771467    6133 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-7b6759dd5d/isvc-sklearn-graph-raw-hpa-a5a79-predictor-5f647f9b68-5t9lb"
Oct 30 22:13:32.793931 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:13:32.793901    6133 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw0-7b6759dd5d/isvc-xgboost-graph-raw-hpa-a5a79-predictor-869d547bd8-f9hpm"]
Oct 30 22:13:32.806390 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:13:32.806360    6133 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-gw0-7b6759dd5d/isvc-xgboost-graph-raw-hpa-a5a79-predictor-869d547bd8-f9hpm"]
Oct 30 22:13:32.911610 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:13:32.911519    6133 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/b184588e-b671-4841-b1fe-ee453ee1116c-proxy-tls\" (UniqueName: \"kubernetes.io/secret/b184588e-b671-4841-b1fe-ee453ee1116c-proxy-tls\") pod \"b184588e-b671-4841-b1fe-ee453ee1116c\" (UID: \"b184588e-b671-4841-b1fe-ee453ee1116c\") "
Oct 30 22:13:32.911610 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:13:32.911586    6133 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b184588e-b671-4841-b1fe-ee453ee1116c-kube-api-access-g2hm8\" (UniqueName: \"kubernetes.io/projected/b184588e-b671-4841-b1fe-ee453ee1116c-kube-api-access-g2hm8\") pod \"b184588e-b671-4841-b1fe-ee453ee1116c\" (UID: \"b184588e-b671-4841-b1fe-ee453ee1116c\") "
Oct 30 22:13:32.911920 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:13:32.911623    6133 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b184588e-b671-4841-b1fe-ee453ee1116c-kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/b184588e-b671-4841-b1fe-ee453ee1116c-kserve-provision-location\") pod \"b184588e-b671-4841-b1fe-ee453ee1116c\" (UID: \"b184588e-b671-4841-b1fe-ee453ee1116c\") "
Oct 30 22:13:32.911920 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:13:32.911666    6133 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b184588e-b671-4841-b1fe-ee453ee1116c-kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/b184588e-b671-4841-b1fe-ee453ee1116c-kube-rbac-proxy-sar-config\") pod \"b184588e-b671-4841-b1fe-ee453ee1116c\" (UID: \"b184588e-b671-4841-b1fe-ee453ee1116c\") "
Oct 30 22:13:32.911920 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:13:32.911691    6133 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b184588e-b671-4841-b1fe-ee453ee1116c-cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/b184588e-b671-4841-b1fe-ee453ee1116c-cabundle-cert\") pod \"b184588e-b671-4841-b1fe-ee453ee1116c\" (UID: \"b184588e-b671-4841-b1fe-ee453ee1116c\") "
Oct 30 22:13:32.912131 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:13:32.911954    6133 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b184588e-b671-4841-b1fe-ee453ee1116c-kserve-provision-location" pod "b184588e-b671-4841-b1fe-ee453ee1116c" (UID: "b184588e-b671-4841-b1fe-ee453ee1116c"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:13:32.912131 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:13:32.912073    6133 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b184588e-b671-4841-b1fe-ee453ee1116c-kube-rbac-proxy-sar-config" pod "b184588e-b671-4841-b1fe-ee453ee1116c" (UID: "b184588e-b671-4841-b1fe-ee453ee1116c"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:13:32.912241 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:13:32.912152    6133 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b184588e-b671-4841-b1fe-ee453ee1116c-cabundle-cert" pod "b184588e-b671-4841-b1fe-ee453ee1116c" (UID: "b184588e-b671-4841-b1fe-ee453ee1116c"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:13:32.913812 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:13:32.913784    6133 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b184588e-b671-4841-b1fe-ee453ee1116c-proxy-tls" pod "b184588e-b671-4841-b1fe-ee453ee1116c" (UID: "b184588e-b671-4841-b1fe-ee453ee1116c"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:13:32.913812 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:13:32.913797    6133 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b184588e-b671-4841-b1fe-ee453ee1116c-kube-api-access-g2hm8" pod "b184588e-b671-4841-b1fe-ee453ee1116c" (UID: "b184588e-b671-4841-b1fe-ee453ee1116c"). InnerVolumeSpecName "kube-api-access-g2hm8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:13:33.012414 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:13:33.012375    6133 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/b184588e-b671-4841-b1fe-ee453ee1116c-kube-rbac-proxy-sar-config\") on node \"ip-10-0-143-246.ec2.internal\" DevicePath \"\""
Oct 30 22:13:33.012414 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:13:33.012409    6133 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/b184588e-b671-4841-b1fe-ee453ee1116c-cabundle-cert\") on node \"ip-10-0-143-246.ec2.internal\" DevicePath \"\""
Oct 30 22:13:33.012414 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:13:33.012420    6133 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/b184588e-b671-4841-b1fe-ee453ee1116c-proxy-tls\") on node \"ip-10-0-143-246.ec2.internal\" DevicePath \"\""
Oct 30 22:13:33.012876 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:13:33.012429    6133 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g2hm8\" (UniqueName: \"kubernetes.io/projected/b184588e-b671-4841-b1fe-ee453ee1116c-kube-api-access-g2hm8\") on node \"ip-10-0-143-246.ec2.internal\" DevicePath \"\""
Oct 30 22:13:33.012876 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:13:33.012439    6133 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/b184588e-b671-4841-b1fe-ee453ee1116c-kserve-provision-location\") on node \"ip-10-0-143-246.ec2.internal\" DevicePath \"\""
Oct 30 22:13:33.354538 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:13:33.354507    6133 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="fbd531c4-ba6f-4632-b6f2-d1fd9dead011" path="/var/lib/kubelet/pods/fbd531c4-ba6f-4632-b6f2-d1fd9dead011/volumes"
Oct 30 22:13:33.734405 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:13:33.734316    6133 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-7b6759dd5d/isvc-sklearn-graph-raw-hpa-a5a79-predictor-5f647f9b68-5t9lb" event={"ID":"b184588e-b671-4841-b1fe-ee453ee1116c","Type":"ContainerDied","Data":"5e9bb39ce79f2a386f1a4bd1ed0f01ae29ebb7fe47d70330502d542886bb6914"}
Oct 30 22:13:33.734405 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:13:33.734346    6133 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-7b6759dd5d/isvc-sklearn-graph-raw-hpa-a5a79-predictor-5f647f9b68-5t9lb"
Oct 30 22:13:33.734405 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:13:33.734387    6133 scope.go:122] "RemoveContainer" containerID="ac51d973832df8a8bd3e0353d200bace409c880698ac56ce5513bf2c087b1c1c"
Oct 30 22:13:33.743362 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:13:33.743309    6133 scope.go:122] "RemoveContainer" containerID="509632e56c5f7a8e7d6a37390635f4bc09f730865b32f6e1965a0169df74c3f2"
Oct 30 22:13:33.751199 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:13:33.751132    6133 scope.go:122] "RemoveContainer" containerID="fdfa80581eb4f8584881b37641a96a2b395f7470ad9ed13a168a11ad9c86a138"
Oct 30 22:13:33.803117 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:13:33.803083    6133 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw0-7b6759dd5d/isvc-sklearn-graph-raw-hpa-a5a79-predictor-5f647f9b68-5t9lb"]
Oct 30 22:13:33.815586 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:13:33.815557    6133 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-gw0-7b6759dd5d/isvc-sklearn-graph-raw-hpa-a5a79-predictor-5f647f9b68-5t9lb"]
Oct 30 22:13:34.927768 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:13:34.927736    6133 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-gw0-7b6759dd5d/message-dumper-raw-f8e55-predictor-7bbf7d9766-68n76"]
Oct 30 22:13:34.928182 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:13:34.928034    6133 cpu_manager.go:414] "containerMap: removing container" podUID="bb80ffe3-13a0-4765-aa4f-4e74eb4f0a42" containerName="kube-rbac-proxy"
Oct 30 22:13:34.928182 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:13:34.928047    6133 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="bb80ffe3-13a0-4765-aa4f-4e74eb4f0a42" containerName="kube-rbac-proxy"
Oct 30 22:13:34.928182 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:13:34.928058    6133 cpu_manager.go:414] "containerMap: removing container" podUID="b184588e-b671-4841-b1fe-ee453ee1116c" containerName="kserve-container"
Oct 30 22:13:34.928182 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:13:34.928063    6133 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b184588e-b671-4841-b1fe-ee453ee1116c" containerName="kserve-container"
Oct 30 22:13:34.928182 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:13:34.928072    6133 cpu_manager.go:414] "containerMap: removing container" podUID="fbd531c4-ba6f-4632-b6f2-d1fd9dead011" containerName="kube-rbac-proxy"
Oct 30 22:13:34.928182 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:13:34.928077    6133 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="fbd531c4-ba6f-4632-b6f2-d1fd9dead011" containerName="kube-rbac-proxy"
Oct 30 22:13:34.928182 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:13:34.928088    6133 cpu_manager.go:414] "containerMap: removing container" podUID="bb80ffe3-13a0-4765-aa4f-4e74eb4f0a42" containerName="kserve-container"
Oct 30 22:13:34.928182 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:13:34.928092    6133 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="bb80ffe3-13a0-4765-aa4f-4e74eb4f0a42" containerName="kserve-container"
Oct 30 22:13:34.928182 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:13:34.928099    6133 cpu_manager.go:414] "containerMap: removing container" podUID="fbd531c4-ba6f-4632-b6f2-d1fd9dead011" containerName="storage-initializer"
Oct 30 22:13:34.928182 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:13:34.928104    6133 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="fbd531c4-ba6f-4632-b6f2-d1fd9dead011" containerName="storage-initializer"
Oct 30 22:13:34.928182 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:13:34.928111    6133 cpu_manager.go:414] "containerMap: removing container" podUID="b184588e-b671-4841-b1fe-ee453ee1116c" containerName="storage-initializer"
Oct 30 22:13:34.928182 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:13:34.928118    6133 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b184588e-b671-4841-b1fe-ee453ee1116c" containerName="storage-initializer"
Oct 30 22:13:34.928182 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:13:34.928127    6133 cpu_manager.go:414] "containerMap: removing container" podUID="bb80ffe3-13a0-4765-aa4f-4e74eb4f0a42" containerName="storage-initializer"
Oct 30 22:13:34.928182 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:13:34.928132    6133 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="bb80ffe3-13a0-4765-aa4f-4e74eb4f0a42" containerName="storage-initializer"
Oct 30 22:13:34.928182 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:13:34.928141    6133 cpu_manager.go:414] "containerMap: removing container" podUID="fbd531c4-ba6f-4632-b6f2-d1fd9dead011" containerName="kserve-container"
Oct 30 22:13:34.928182 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:13:34.928146    6133 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="fbd531c4-ba6f-4632-b6f2-d1fd9dead011" containerName="kserve-container"
Oct 30 22:13:34.928182 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:13:34.928151    6133 cpu_manager.go:414] "containerMap: removing container" podUID="b184588e-b671-4841-b1fe-ee453ee1116c" containerName="kube-rbac-proxy"
Oct 30 22:13:34.928182 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:13:34.928156    6133 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b184588e-b671-4841-b1fe-ee453ee1116c" containerName="kube-rbac-proxy"
Oct 30 22:13:34.928696 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:13:34.928196    6133 memory_manager.go:365] "RemoveStaleState removing state" podUID="b184588e-b671-4841-b1fe-ee453ee1116c" containerName="kserve-container"
Oct 30 22:13:34.928696 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:13:34.928203    6133 memory_manager.go:365] "RemoveStaleState removing state" podUID="fbd531c4-ba6f-4632-b6f2-d1fd9dead011" containerName="kube-rbac-proxy"
Oct 30 22:13:34.928696 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:13:34.928210    6133 memory_manager.go:365] "RemoveStaleState removing state" podUID="bb80ffe3-13a0-4765-aa4f-4e74eb4f0a42" containerName="kserve-container"
Oct 30 22:13:34.928696 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:13:34.928218    6133 memory_manager.go:365] "RemoveStaleState removing state" podUID="fbd531c4-ba6f-4632-b6f2-d1fd9dead011" containerName="kserve-container"
Oct 30 22:13:34.928696 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:13:34.928224    6133 memory_manager.go:365] "RemoveStaleState removing state" podUID="b184588e-b671-4841-b1fe-ee453ee1116c" containerName="kube-rbac-proxy"
Oct 30 22:13:34.928696 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:13:34.928231    6133 memory_manager.go:365] "RemoveStaleState removing state" podUID="bb80ffe3-13a0-4765-aa4f-4e74eb4f0a42" containerName="kube-rbac-proxy"
Oct 30 22:13:34.932724 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:13:34.932706    6133 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-7b6759dd5d/message-dumper-raw-f8e55-predictor-7bbf7d9766-68n76"
Oct 30 22:13:34.960596 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:13:34.960557    6133 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw0-7b6759dd5d\"/\"openshift-service-ca.crt\""
Oct 30 22:13:34.960596 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:13:34.960557    6133 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw0-7b6759dd5d\"/\"message-dumper-raw-f8e55-predictor-serving-cert\""
Oct 30 22:13:34.960596 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:13:34.960570    6133 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw0-7b6759dd5d\"/\"kube-root-ca.crt\""
Oct 30 22:13:34.960838 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:13:34.960570    6133 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw0-7b6759dd5d\"/\"message-dumper-raw-f8e55-kube-rbac-proxy-sar-config\""
Oct 30 22:13:34.960838 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:13:34.960646    6133 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw0-7b6759dd5d\"/\"default-dockercfg-4b7xh\""
Oct 30 22:13:34.961595 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:13:34.961579    6133 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw0-7b6759dd5d/message-dumper-raw-f8e55-predictor-7bbf7d9766-68n76"]
Oct 30 22:13:35.129092 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:13:35.129054    6133 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/be61d92f-8af7-40cd-82f4-3d4354ad7e1a-proxy-tls\") pod \"message-dumper-raw-f8e55-predictor-7bbf7d9766-68n76\" (UID: \"be61d92f-8af7-40cd-82f4-3d4354ad7e1a\") " pod="e2e-gw0-7b6759dd5d/message-dumper-raw-f8e55-predictor-7bbf7d9766-68n76"
Oct 30 22:13:35.129092 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:13:35.129091    6133 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/be61d92f-8af7-40cd-82f4-3d4354ad7e1a-kube-rbac-proxy-sar-config\") pod \"message-dumper-raw-f8e55-predictor-7bbf7d9766-68n76\" (UID: \"be61d92f-8af7-40cd-82f4-3d4354ad7e1a\") " pod="e2e-gw0-7b6759dd5d/message-dumper-raw-f8e55-predictor-7bbf7d9766-68n76"
Oct 30 22:13:35.129309 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:13:35.129122    6133 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9cvf6\" (UniqueName: \"kubernetes.io/projected/be61d92f-8af7-40cd-82f4-3d4354ad7e1a-kube-api-access-9cvf6\") pod \"message-dumper-raw-f8e55-predictor-7bbf7d9766-68n76\" (UID: \"be61d92f-8af7-40cd-82f4-3d4354ad7e1a\") " pod="e2e-gw0-7b6759dd5d/message-dumper-raw-f8e55-predictor-7bbf7d9766-68n76"
Oct 30 22:13:35.230090 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:13:35.229981    6133 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9cvf6\" (UniqueName: \"kubernetes.io/projected/be61d92f-8af7-40cd-82f4-3d4354ad7e1a-kube-api-access-9cvf6\") pod \"message-dumper-raw-f8e55-predictor-7bbf7d9766-68n76\" (UID: \"be61d92f-8af7-40cd-82f4-3d4354ad7e1a\") " pod="e2e-gw0-7b6759dd5d/message-dumper-raw-f8e55-predictor-7bbf7d9766-68n76"
Oct 30 22:13:35.230233 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:13:35.230099    6133 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/be61d92f-8af7-40cd-82f4-3d4354ad7e1a-proxy-tls\") pod \"message-dumper-raw-f8e55-predictor-7bbf7d9766-68n76\" (UID: \"be61d92f-8af7-40cd-82f4-3d4354ad7e1a\") " pod="e2e-gw0-7b6759dd5d/message-dumper-raw-f8e55-predictor-7bbf7d9766-68n76"
Oct 30 22:13:35.230233 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:13:35.230128    6133 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/be61d92f-8af7-40cd-82f4-3d4354ad7e1a-kube-rbac-proxy-sar-config\") pod \"message-dumper-raw-f8e55-predictor-7bbf7d9766-68n76\" (UID: \"be61d92f-8af7-40cd-82f4-3d4354ad7e1a\") " pod="e2e-gw0-7b6759dd5d/message-dumper-raw-f8e55-predictor-7bbf7d9766-68n76"
Oct 30 22:13:35.230781 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:13:35.230752    6133 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/be61d92f-8af7-40cd-82f4-3d4354ad7e1a-kube-rbac-proxy-sar-config\") pod \"message-dumper-raw-f8e55-predictor-7bbf7d9766-68n76\" (UID: \"be61d92f-8af7-40cd-82f4-3d4354ad7e1a\") " pod="e2e-gw0-7b6759dd5d/message-dumper-raw-f8e55-predictor-7bbf7d9766-68n76"
Oct 30 22:13:35.232809 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:13:35.232779    6133 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/be61d92f-8af7-40cd-82f4-3d4354ad7e1a-proxy-tls\") pod \"message-dumper-raw-f8e55-predictor-7bbf7d9766-68n76\" (UID: \"be61d92f-8af7-40cd-82f4-3d4354ad7e1a\") " pod="e2e-gw0-7b6759dd5d/message-dumper-raw-f8e55-predictor-7bbf7d9766-68n76"
Oct 30 22:13:35.251181 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:13:35.251145    6133 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9cvf6\" (UniqueName: \"kubernetes.io/projected/be61d92f-8af7-40cd-82f4-3d4354ad7e1a-kube-api-access-9cvf6\") pod \"message-dumper-raw-f8e55-predictor-7bbf7d9766-68n76\" (UID: \"be61d92f-8af7-40cd-82f4-3d4354ad7e1a\") " pod="e2e-gw0-7b6759dd5d/message-dumper-raw-f8e55-predictor-7bbf7d9766-68n76"
Oct 30 22:13:35.354718 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:13:35.354682    6133 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b184588e-b671-4841-b1fe-ee453ee1116c" path="/var/lib/kubelet/pods/b184588e-b671-4841-b1fe-ee453ee1116c/volumes"
Oct 30 22:13:35.547141 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:13:35.547103    6133 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-7b6759dd5d/message-dumper-raw-f8e55-predictor-7bbf7d9766-68n76"
Oct 30 22:13:35.840462 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:13:35.840425    6133 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw0-7b6759dd5d/message-dumper-raw-f8e55-predictor-7bbf7d9766-68n76"]
Oct 30 22:13:35.841973 ip-10-0-143-246 kubenswrapper[6133]: W1030 22:13:35.841937    6133 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbe61d92f_8af7_40cd_82f4_3d4354ad7e1a.slice/crio-6722077e1c6fb03811bbc3bf783935f8e3b80489ecf11769afe7cc15f2930a9f WatchSource:0}: Error finding container 6722077e1c6fb03811bbc3bf783935f8e3b80489ecf11769afe7cc15f2930a9f: Status 404 returned error can't find the container with id 6722077e1c6fb03811bbc3bf783935f8e3b80489ecf11769afe7cc15f2930a9f
Oct 30 22:13:35.843958 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:13:35.843942    6133 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 22:13:36.747009 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:13:36.746950    6133 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-7b6759dd5d/message-dumper-raw-f8e55-predictor-7bbf7d9766-68n76" event={"ID":"be61d92f-8af7-40cd-82f4-3d4354ad7e1a","Type":"ContainerStarted","Data":"6722077e1c6fb03811bbc3bf783935f8e3b80489ecf11769afe7cc15f2930a9f"}
Oct 30 22:13:37.753211 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:13:37.753173    6133 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-7b6759dd5d/message-dumper-raw-f8e55-predictor-7bbf7d9766-68n76" event={"ID":"be61d92f-8af7-40cd-82f4-3d4354ad7e1a","Type":"ContainerStarted","Data":"c9321c5d3e0b75c5d28faec941969912ef79a7585bbc96cc9c13bac013af0085"}
Oct 30 22:13:37.753637 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:13:37.753242    6133 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw0-7b6759dd5d/message-dumper-raw-f8e55-predictor-7bbf7d9766-68n76"
Oct 30 22:13:37.753637 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:13:37.753265    6133 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-7b6759dd5d/message-dumper-raw-f8e55-predictor-7bbf7d9766-68n76" event={"ID":"be61d92f-8af7-40cd-82f4-3d4354ad7e1a","Type":"ContainerStarted","Data":"1d077c531610c47a816b61c3e782e2c29b42bcb93a730ad8f35a481532207566"}
Oct 30 22:13:37.753637 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:13:37.753412    6133 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw0-7b6759dd5d/message-dumper-raw-f8e55-predictor-7bbf7d9766-68n76"
Oct 30 22:13:37.754900 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:13:37.754881    6133 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw0-7b6759dd5d/message-dumper-raw-f8e55-predictor-7bbf7d9766-68n76"
Oct 30 22:13:37.796423 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:13:37.796366    6133 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-gw0-7b6759dd5d/message-dumper-raw-f8e55-predictor-7bbf7d9766-68n76" podStartSLOduration=2.328706644 podStartE2EDuration="3.796353699s" totalImagesPullingTime="1.467647055s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:13:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:13:37.795658219 +0000 UTC m=+1124.556414107" watchObservedRunningTime="2026-10-30 22:13:37.796353699 +0000 UTC m=+1124.557109582"
Oct 30 22:13:39.526762 ip-10-0-143-246 kubenswrapper[6133]: E1030 22:13:39.526730    6133 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6jppj" containerName="extractor"
Oct 30 22:13:44.769104 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:13:44.769071    6133 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw0-7b6759dd5d/message-dumper-raw-f8e55-predictor-7bbf7d9766-68n76"
Oct 30 22:13:53.467064 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:13:53.467032    6133 scope.go:122] "RemoveContainer" containerID="215b67b68b881759f2ad3eedf42e532096c674e0e22d99bec8d6fd8b968be788"
Oct 30 22:13:53.474950 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:13:53.474932    6133 scope.go:122] "RemoveContainer" containerID="0a60bf4fb900b85fd05b56caf63f539b4ad305ffc1c9a1b550ef39c0bc85ea05"
Oct 30 22:13:53.482535 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:13:53.482510    6133 scope.go:122] "RemoveContainer" containerID="1fd68968c7c8bc62402be1ef466c06389e3aec6ef8c140d2f52fdaa9a5a2a828"
Oct 30 22:13:55.178578 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:13:55.178541    6133 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-gw0-7b6759dd5d/isvc-logger-raw-f8e55-predictor-845877cbb-jmr69"]
Oct 30 22:13:55.184145 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:13:55.184117    6133 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-7b6759dd5d/isvc-logger-raw-f8e55-predictor-845877cbb-jmr69"
Oct 30 22:13:55.200951 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:13:55.200926    6133 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw0-7b6759dd5d\"/\"isvc-logger-raw-f8e55-kube-rbac-proxy-sar-config\""
Oct 30 22:13:55.201117 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:13:55.201062    6133 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw0-7b6759dd5d\"/\"odh-kserve-custom-ca-bundle\""
Oct 30 22:13:55.205263 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:13:55.205245    6133 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw0-7b6759dd5d\"/\"seaweedfs-s3-creds\""
Oct 30 22:13:55.212689 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:13:55.212662    6133 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw0-7b6759dd5d\"/\"isvc-logger-raw-f8e55-predictor-serving-cert\""
Oct 30 22:13:55.213594 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:13:55.213568    6133 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw0-7b6759dd5d/isvc-logger-raw-f8e55-predictor-845877cbb-jmr69"]
Oct 30 22:13:55.285610 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:13:55.285576    6133 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-csvbz\" (UniqueName: \"kubernetes.io/projected/6a44df26-dc83-4b1b-94ba-fd9464a7e3e5-kube-api-access-csvbz\") pod \"isvc-logger-raw-f8e55-predictor-845877cbb-jmr69\" (UID: \"6a44df26-dc83-4b1b-94ba-fd9464a7e3e5\") " pod="e2e-gw0-7b6759dd5d/isvc-logger-raw-f8e55-predictor-845877cbb-jmr69"
Oct 30 22:13:55.285823 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:13:55.285627    6133 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/6a44df26-dc83-4b1b-94ba-fd9464a7e3e5-cabundle-cert\") pod \"isvc-logger-raw-f8e55-predictor-845877cbb-jmr69\" (UID: \"6a44df26-dc83-4b1b-94ba-fd9464a7e3e5\") " pod="e2e-gw0-7b6759dd5d/isvc-logger-raw-f8e55-predictor-845877cbb-jmr69"
Oct 30 22:13:55.285823 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:13:55.285653    6133 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/6a44df26-dc83-4b1b-94ba-fd9464a7e3e5-kserve-provision-location\") pod \"isvc-logger-raw-f8e55-predictor-845877cbb-jmr69\" (UID: \"6a44df26-dc83-4b1b-94ba-fd9464a7e3e5\") " pod="e2e-gw0-7b6759dd5d/isvc-logger-raw-f8e55-predictor-845877cbb-jmr69"
Oct 30 22:13:55.285823 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:13:55.285808    6133 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/6a44df26-dc83-4b1b-94ba-fd9464a7e3e5-kube-rbac-proxy-sar-config\") pod \"isvc-logger-raw-f8e55-predictor-845877cbb-jmr69\" (UID: \"6a44df26-dc83-4b1b-94ba-fd9464a7e3e5\") " pod="e2e-gw0-7b6759dd5d/isvc-logger-raw-f8e55-predictor-845877cbb-jmr69"
Oct 30 22:13:55.286047 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:13:55.285849    6133 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/6a44df26-dc83-4b1b-94ba-fd9464a7e3e5-proxy-tls\") pod \"isvc-logger-raw-f8e55-predictor-845877cbb-jmr69\" (UID: \"6a44df26-dc83-4b1b-94ba-fd9464a7e3e5\") " pod="e2e-gw0-7b6759dd5d/isvc-logger-raw-f8e55-predictor-845877cbb-jmr69"
Oct 30 22:13:55.386493 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:13:55.386459    6133 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/6a44df26-dc83-4b1b-94ba-fd9464a7e3e5-cabundle-cert\") pod \"isvc-logger-raw-f8e55-predictor-845877cbb-jmr69\" (UID: \"6a44df26-dc83-4b1b-94ba-fd9464a7e3e5\") " pod="e2e-gw0-7b6759dd5d/isvc-logger-raw-f8e55-predictor-845877cbb-jmr69"
Oct 30 22:13:55.386670 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:13:55.386500    6133 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/6a44df26-dc83-4b1b-94ba-fd9464a7e3e5-kserve-provision-location\") pod \"isvc-logger-raw-f8e55-predictor-845877cbb-jmr69\" (UID: \"6a44df26-dc83-4b1b-94ba-fd9464a7e3e5\") " pod="e2e-gw0-7b6759dd5d/isvc-logger-raw-f8e55-predictor-845877cbb-jmr69"
Oct 30 22:13:55.386670 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:13:55.386536    6133 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/6a44df26-dc83-4b1b-94ba-fd9464a7e3e5-kube-rbac-proxy-sar-config\") pod \"isvc-logger-raw-f8e55-predictor-845877cbb-jmr69\" (UID: \"6a44df26-dc83-4b1b-94ba-fd9464a7e3e5\") " pod="e2e-gw0-7b6759dd5d/isvc-logger-raw-f8e55-predictor-845877cbb-jmr69"
Oct 30 22:13:55.386670 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:13:55.386562    6133 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/6a44df26-dc83-4b1b-94ba-fd9464a7e3e5-proxy-tls\") pod \"isvc-logger-raw-f8e55-predictor-845877cbb-jmr69\" (UID: \"6a44df26-dc83-4b1b-94ba-fd9464a7e3e5\") " pod="e2e-gw0-7b6759dd5d/isvc-logger-raw-f8e55-predictor-845877cbb-jmr69"
Oct 30 22:13:55.386670 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:13:55.386628    6133 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-csvbz\" (UniqueName: \"kubernetes.io/projected/6a44df26-dc83-4b1b-94ba-fd9464a7e3e5-kube-api-access-csvbz\") pod \"isvc-logger-raw-f8e55-predictor-845877cbb-jmr69\" (UID: \"6a44df26-dc83-4b1b-94ba-fd9464a7e3e5\") " pod="e2e-gw0-7b6759dd5d/isvc-logger-raw-f8e55-predictor-845877cbb-jmr69"
Oct 30 22:13:55.387447 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:13:55.387415    6133 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/6a44df26-dc83-4b1b-94ba-fd9464a7e3e5-kserve-provision-location\") pod \"isvc-logger-raw-f8e55-predictor-845877cbb-jmr69\" (UID: \"6a44df26-dc83-4b1b-94ba-fd9464a7e3e5\") " pod="e2e-gw0-7b6759dd5d/isvc-logger-raw-f8e55-predictor-845877cbb-jmr69"
Oct 30 22:13:55.387755 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:13:55.387726    6133 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/6a44df26-dc83-4b1b-94ba-fd9464a7e3e5-cabundle-cert\") pod \"isvc-logger-raw-f8e55-predictor-845877cbb-jmr69\" (UID: \"6a44df26-dc83-4b1b-94ba-fd9464a7e3e5\") " pod="e2e-gw0-7b6759dd5d/isvc-logger-raw-f8e55-predictor-845877cbb-jmr69"
Oct 30 22:13:55.387915 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:13:55.387822    6133 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/6a44df26-dc83-4b1b-94ba-fd9464a7e3e5-kube-rbac-proxy-sar-config\") pod \"isvc-logger-raw-f8e55-predictor-845877cbb-jmr69\" (UID: \"6a44df26-dc83-4b1b-94ba-fd9464a7e3e5\") " pod="e2e-gw0-7b6759dd5d/isvc-logger-raw-f8e55-predictor-845877cbb-jmr69"
Oct 30 22:13:55.390193 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:13:55.390169    6133 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/6a44df26-dc83-4b1b-94ba-fd9464a7e3e5-proxy-tls\") pod \"isvc-logger-raw-f8e55-predictor-845877cbb-jmr69\" (UID: \"6a44df26-dc83-4b1b-94ba-fd9464a7e3e5\") " pod="e2e-gw0-7b6759dd5d/isvc-logger-raw-f8e55-predictor-845877cbb-jmr69"
Oct 30 22:13:55.408611 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:13:55.408581    6133 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-csvbz\" (UniqueName: \"kubernetes.io/projected/6a44df26-dc83-4b1b-94ba-fd9464a7e3e5-kube-api-access-csvbz\") pod \"isvc-logger-raw-f8e55-predictor-845877cbb-jmr69\" (UID: \"6a44df26-dc83-4b1b-94ba-fd9464a7e3e5\") " pod="e2e-gw0-7b6759dd5d/isvc-logger-raw-f8e55-predictor-845877cbb-jmr69"
Oct 30 22:13:55.504965 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:13:55.504927    6133 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-7b6759dd5d/isvc-logger-raw-f8e55-predictor-845877cbb-jmr69"
Oct 30 22:13:55.819735 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:13:55.819711    6133 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw0-7b6759dd5d/isvc-logger-raw-f8e55-predictor-845877cbb-jmr69"]
Oct 30 22:13:55.820982 ip-10-0-143-246 kubenswrapper[6133]: W1030 22:13:55.820955    6133 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6a44df26_dc83_4b1b_94ba_fd9464a7e3e5.slice/crio-5c83c4edd76c3b893391ea04660ce0869a6c2bca5212e9af1ccebdc883063c47 WatchSource:0}: Error finding container 5c83c4edd76c3b893391ea04660ce0869a6c2bca5212e9af1ccebdc883063c47: Status 404 returned error can't find the container with id 5c83c4edd76c3b893391ea04660ce0869a6c2bca5212e9af1ccebdc883063c47
Oct 30 22:13:56.829055 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:13:56.829006    6133 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-7b6759dd5d/isvc-logger-raw-f8e55-predictor-845877cbb-jmr69" event={"ID":"6a44df26-dc83-4b1b-94ba-fd9464a7e3e5","Type":"ContainerStarted","Data":"8ecd485ecaab338dc63e19ea6dec0997fab3f47dc134975200508788fc3ba283"}
Oct 30 22:13:56.829556 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:13:56.829081    6133 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-7b6759dd5d/isvc-logger-raw-f8e55-predictor-845877cbb-jmr69" event={"ID":"6a44df26-dc83-4b1b-94ba-fd9464a7e3e5","Type":"ContainerStarted","Data":"5c83c4edd76c3b893391ea04660ce0869a6c2bca5212e9af1ccebdc883063c47"}
Oct 30 22:13:58.514955 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:13:58.514917    6133 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-gw1-2e5a75ca98/isvc-secondary-795342-predictor-58fc4fd76b-lpb6n"]
Oct 30 22:13:58.518243 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:13:58.518215    6133 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-2e5a75ca98/isvc-secondary-795342-predictor-58fc4fd76b-lpb6n"
Oct 30 22:13:58.534700 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:13:58.534675    6133 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw1-2e5a75ca98\"/\"kube-root-ca.crt\""
Oct 30 22:13:58.546354 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:13:58.546331    6133 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw1-2e5a75ca98\"/\"isvc-secondary-795342-kube-rbac-proxy-sar-config\""
Oct 30 22:13:58.546354 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:13:58.546360    6133 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw1-2e5a75ca98\"/\"invalid-s3-sa-795342-dockercfg-vn49f\""
Oct 30 22:13:58.546519 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:13:58.546374    6133 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw1-2e5a75ca98\"/\"invalid-s3-secret-795342\""
Oct 30 22:13:58.546519 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:13:58.546394    6133 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw1-2e5a75ca98\"/\"openshift-service-ca.crt\""
Oct 30 22:13:58.546519 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:13:58.546396    6133 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw1-2e5a75ca98\"/\"odh-kserve-custom-ca-bundle\""
Oct 30 22:13:58.546519 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:13:58.546484    6133 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw1-2e5a75ca98\"/\"isvc-secondary-795342-predictor-serving-cert\""
Oct 30 22:13:58.547888 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:13:58.547867    6133 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw1-2e5a75ca98/isvc-secondary-795342-predictor-58fc4fd76b-lpb6n"]
Oct 30 22:13:58.612821 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:13:58.612782    6133 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/c5487b74-692c-419b-8d07-c7074b6cdb7e-proxy-tls\") pod \"isvc-secondary-795342-predictor-58fc4fd76b-lpb6n\" (UID: \"c5487b74-692c-419b-8d07-c7074b6cdb7e\") " pod="e2e-gw1-2e5a75ca98/isvc-secondary-795342-predictor-58fc4fd76b-lpb6n"
Oct 30 22:13:58.612821 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:13:58.612826    6133 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/c5487b74-692c-419b-8d07-c7074b6cdb7e-kserve-provision-location\") pod \"isvc-secondary-795342-predictor-58fc4fd76b-lpb6n\" (UID: \"c5487b74-692c-419b-8d07-c7074b6cdb7e\") " pod="e2e-gw1-2e5a75ca98/isvc-secondary-795342-predictor-58fc4fd76b-lpb6n"
Oct 30 22:13:58.613075 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:13:58.612912    6133 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/c5487b74-692c-419b-8d07-c7074b6cdb7e-kube-rbac-proxy-sar-config\") pod \"isvc-secondary-795342-predictor-58fc4fd76b-lpb6n\" (UID: \"c5487b74-692c-419b-8d07-c7074b6cdb7e\") " pod="e2e-gw1-2e5a75ca98/isvc-secondary-795342-predictor-58fc4fd76b-lpb6n"
Oct 30 22:13:58.613075 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:13:58.613056    6133 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qf7bd\" (UniqueName: \"kubernetes.io/projected/c5487b74-692c-419b-8d07-c7074b6cdb7e-kube-api-access-qf7bd\") pod \"isvc-secondary-795342-predictor-58fc4fd76b-lpb6n\" (UID: \"c5487b74-692c-419b-8d07-c7074b6cdb7e\") " pod="e2e-gw1-2e5a75ca98/isvc-secondary-795342-predictor-58fc4fd76b-lpb6n"
Oct 30 22:13:58.613142 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:13:58.613085    6133 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/c5487b74-692c-419b-8d07-c7074b6cdb7e-cabundle-cert\") pod \"isvc-secondary-795342-predictor-58fc4fd76b-lpb6n\" (UID: \"c5487b74-692c-419b-8d07-c7074b6cdb7e\") " pod="e2e-gw1-2e5a75ca98/isvc-secondary-795342-predictor-58fc4fd76b-lpb6n"
Oct 30 22:13:58.714464 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:13:58.714430    6133 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/c5487b74-692c-419b-8d07-c7074b6cdb7e-proxy-tls\") pod \"isvc-secondary-795342-predictor-58fc4fd76b-lpb6n\" (UID: \"c5487b74-692c-419b-8d07-c7074b6cdb7e\") " pod="e2e-gw1-2e5a75ca98/isvc-secondary-795342-predictor-58fc4fd76b-lpb6n"
Oct 30 22:13:58.714648 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:13:58.714473    6133 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/c5487b74-692c-419b-8d07-c7074b6cdb7e-kserve-provision-location\") pod \"isvc-secondary-795342-predictor-58fc4fd76b-lpb6n\" (UID: \"c5487b74-692c-419b-8d07-c7074b6cdb7e\") " pod="e2e-gw1-2e5a75ca98/isvc-secondary-795342-predictor-58fc4fd76b-lpb6n"
Oct 30 22:13:58.714648 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:13:58.714504    6133 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/c5487b74-692c-419b-8d07-c7074b6cdb7e-kube-rbac-proxy-sar-config\") pod \"isvc-secondary-795342-predictor-58fc4fd76b-lpb6n\" (UID: \"c5487b74-692c-419b-8d07-c7074b6cdb7e\") " pod="e2e-gw1-2e5a75ca98/isvc-secondary-795342-predictor-58fc4fd76b-lpb6n"
Oct 30 22:13:58.714648 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:13:58.714535    6133 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qf7bd\" (UniqueName: \"kubernetes.io/projected/c5487b74-692c-419b-8d07-c7074b6cdb7e-kube-api-access-qf7bd\") pod \"isvc-secondary-795342-predictor-58fc4fd76b-lpb6n\" (UID: \"c5487b74-692c-419b-8d07-c7074b6cdb7e\") " pod="e2e-gw1-2e5a75ca98/isvc-secondary-795342-predictor-58fc4fd76b-lpb6n"
Oct 30 22:13:58.714648 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:13:58.714557    6133 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/c5487b74-692c-419b-8d07-c7074b6cdb7e-cabundle-cert\") pod \"isvc-secondary-795342-predictor-58fc4fd76b-lpb6n\" (UID: \"c5487b74-692c-419b-8d07-c7074b6cdb7e\") " pod="e2e-gw1-2e5a75ca98/isvc-secondary-795342-predictor-58fc4fd76b-lpb6n"
Oct 30 22:13:58.715077 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:13:58.715058    6133 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/c5487b74-692c-419b-8d07-c7074b6cdb7e-kserve-provision-location\") pod \"isvc-secondary-795342-predictor-58fc4fd76b-lpb6n\" (UID: \"c5487b74-692c-419b-8d07-c7074b6cdb7e\") " pod="e2e-gw1-2e5a75ca98/isvc-secondary-795342-predictor-58fc4fd76b-lpb6n"
Oct 30 22:13:58.715318 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:13:58.715299    6133 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/c5487b74-692c-419b-8d07-c7074b6cdb7e-cabundle-cert\") pod \"isvc-secondary-795342-predictor-58fc4fd76b-lpb6n\" (UID: \"c5487b74-692c-419b-8d07-c7074b6cdb7e\") " pod="e2e-gw1-2e5a75ca98/isvc-secondary-795342-predictor-58fc4fd76b-lpb6n"
Oct 30 22:13:58.715388 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:13:58.715303    6133 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/c5487b74-692c-419b-8d07-c7074b6cdb7e-kube-rbac-proxy-sar-config\") pod \"isvc-secondary-795342-predictor-58fc4fd76b-lpb6n\" (UID: \"c5487b74-692c-419b-8d07-c7074b6cdb7e\") " pod="e2e-gw1-2e5a75ca98/isvc-secondary-795342-predictor-58fc4fd76b-lpb6n"
Oct 30 22:13:58.717007 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:13:58.716970    6133 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/c5487b74-692c-419b-8d07-c7074b6cdb7e-proxy-tls\") pod \"isvc-secondary-795342-predictor-58fc4fd76b-lpb6n\" (UID: \"c5487b74-692c-419b-8d07-c7074b6cdb7e\") " pod="e2e-gw1-2e5a75ca98/isvc-secondary-795342-predictor-58fc4fd76b-lpb6n"
Oct 30 22:13:58.735500 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:13:58.735471    6133 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qf7bd\" (UniqueName: \"kubernetes.io/projected/c5487b74-692c-419b-8d07-c7074b6cdb7e-kube-api-access-qf7bd\") pod \"isvc-secondary-795342-predictor-58fc4fd76b-lpb6n\" (UID: \"c5487b74-692c-419b-8d07-c7074b6cdb7e\") " pod="e2e-gw1-2e5a75ca98/isvc-secondary-795342-predictor-58fc4fd76b-lpb6n"
Oct 30 22:13:58.834747 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:13:58.834667    6133 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-2e5a75ca98/isvc-secondary-795342-predictor-58fc4fd76b-lpb6n"
Oct 30 22:13:59.141472 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:13:59.141447    6133 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw1-2e5a75ca98/isvc-secondary-795342-predictor-58fc4fd76b-lpb6n"]
Oct 30 22:13:59.143973 ip-10-0-143-246 kubenswrapper[6133]: W1030 22:13:59.143940    6133 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc5487b74_692c_419b_8d07_c7074b6cdb7e.slice/crio-39492b404f5e05be88106016a9dcff7d1afda73fcabdfa45505572c75db4b52e WatchSource:0}: Error finding container 39492b404f5e05be88106016a9dcff7d1afda73fcabdfa45505572c75db4b52e: Status 404 returned error can't find the container with id 39492b404f5e05be88106016a9dcff7d1afda73fcabdfa45505572c75db4b52e
Oct 30 22:13:59.842894 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:13:59.842857    6133 generic.go:356] "Generic (PLEG): container finished" podID="6a44df26-dc83-4b1b-94ba-fd9464a7e3e5" containerID="8ecd485ecaab338dc63e19ea6dec0997fab3f47dc134975200508788fc3ba283" exitCode=0
Oct 30 22:13:59.843380 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:13:59.842924    6133 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-7b6759dd5d/isvc-logger-raw-f8e55-predictor-845877cbb-jmr69" event={"ID":"6a44df26-dc83-4b1b-94ba-fd9464a7e3e5","Type":"ContainerDied","Data":"8ecd485ecaab338dc63e19ea6dec0997fab3f47dc134975200508788fc3ba283"}
Oct 30 22:13:59.845022 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:13:59.844978    6133 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-2e5a75ca98/isvc-secondary-795342-predictor-58fc4fd76b-lpb6n" event={"ID":"c5487b74-692c-419b-8d07-c7074b6cdb7e","Type":"ContainerStarted","Data":"86d79169e8f01efff54d0af7dd5bbf98f7226f939f33dd3608ba619925d72da6"}
Oct 30 22:13:59.845096 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:13:59.845059    6133 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-2e5a75ca98/isvc-secondary-795342-predictor-58fc4fd76b-lpb6n" event={"ID":"c5487b74-692c-419b-8d07-c7074b6cdb7e","Type":"ContainerStarted","Data":"39492b404f5e05be88106016a9dcff7d1afda73fcabdfa45505572c75db4b52e"}
Oct 30 22:14:00.850683 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:14:00.850644    6133 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-7b6759dd5d/isvc-logger-raw-f8e55-predictor-845877cbb-jmr69" event={"ID":"6a44df26-dc83-4b1b-94ba-fd9464a7e3e5","Type":"ContainerStarted","Data":"56c96fb5c2db55e02a683ed5aafdc56fa4d8a9116f3f2f63db25bdb7d2c3eccc"}
Oct 30 22:14:00.851074 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:14:00.850734    6133 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-7b6759dd5d/isvc-logger-raw-f8e55-predictor-845877cbb-jmr69" event={"ID":"6a44df26-dc83-4b1b-94ba-fd9464a7e3e5","Type":"ContainerStarted","Data":"502db0bdd6da1dc299bea2bc3a46b3dd59af9954385e38292a4f09e82666966c"}
Oct 30 22:14:00.851074 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:14:00.850775    6133 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-7b6759dd5d/isvc-logger-raw-f8e55-predictor-845877cbb-jmr69" event={"ID":"6a44df26-dc83-4b1b-94ba-fd9464a7e3e5","Type":"ContainerStarted","Data":"cb638c5b34fff17250c4daeda2386cdde72b6551c69a2235aeeef9dce795664b"}
Oct 30 22:14:00.851208 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:14:00.851188    6133 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw0-7b6759dd5d/isvc-logger-raw-f8e55-predictor-845877cbb-jmr69"
Oct 30 22:14:00.851267 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:14:00.851257    6133 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw0-7b6759dd5d/isvc-logger-raw-f8e55-predictor-845877cbb-jmr69"
Oct 30 22:14:00.851304 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:14:00.851294    6133 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw0-7b6759dd5d/isvc-logger-raw-f8e55-predictor-845877cbb-jmr69"
Oct 30 22:14:00.852716 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:14:00.852687    6133 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7b6759dd5d/isvc-logger-raw-f8e55-predictor-845877cbb-jmr69" podUID="6a44df26-dc83-4b1b-94ba-fd9464a7e3e5" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.27:8080: connect: connection refused"
Oct 30 22:14:00.853431 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:14:00.853395    6133 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7b6759dd5d/isvc-logger-raw-f8e55-predictor-845877cbb-jmr69" podUID="6a44df26-dc83-4b1b-94ba-fd9464a7e3e5" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 22:14:00.911068 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:14:00.910984    6133 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-gw0-7b6759dd5d/isvc-logger-raw-f8e55-predictor-845877cbb-jmr69" podStartSLOduration=2.205480855 podStartE2EDuration="5.910968745s" totalImagesPullingTime="0s" totalInitContainerRuntime="3.70548789s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:13:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:14:00.908425239 +0000 UTC m=+1147.669181142" watchObservedRunningTime="2026-10-30 22:14:00.910968745 +0000 UTC m=+1147.671724631"
Oct 30 22:14:01.855233 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:14:01.855184    6133 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7b6759dd5d/isvc-logger-raw-f8e55-predictor-845877cbb-jmr69" podUID="6a44df26-dc83-4b1b-94ba-fd9464a7e3e5" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.27:8080: connect: connection refused"
Oct 30 22:14:01.855685 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:14:01.855620    6133 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7b6759dd5d/isvc-logger-raw-f8e55-predictor-845877cbb-jmr69" podUID="6a44df26-dc83-4b1b-94ba-fd9464a7e3e5" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 22:14:05.876453 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:14:05.876421    6133 log.go:25] "Finished parsing log file" path="/var/log/pods/e2e-gw1-2e5a75ca98_isvc-secondary-795342-predictor-58fc4fd76b-lpb6n_c5487b74-692c-419b-8d07-c7074b6cdb7e/storage-initializer/0.log"
Oct 30 22:14:05.876851 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:14:05.876464    6133 generic.go:356] "Generic (PLEG): container finished" podID="c5487b74-692c-419b-8d07-c7074b6cdb7e" containerID="86d79169e8f01efff54d0af7dd5bbf98f7226f939f33dd3608ba619925d72da6" exitCode=1
Oct 30 22:14:05.876851 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:14:05.876507    6133 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-2e5a75ca98/isvc-secondary-795342-predictor-58fc4fd76b-lpb6n" event={"ID":"c5487b74-692c-419b-8d07-c7074b6cdb7e","Type":"ContainerDied","Data":"86d79169e8f01efff54d0af7dd5bbf98f7226f939f33dd3608ba619925d72da6"}
Oct 30 22:14:06.860111 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:14:06.860077    6133 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw0-7b6759dd5d/isvc-logger-raw-f8e55-predictor-845877cbb-jmr69"
Oct 30 22:14:06.860718 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:14:06.860682    6133 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7b6759dd5d/isvc-logger-raw-f8e55-predictor-845877cbb-jmr69" podUID="6a44df26-dc83-4b1b-94ba-fd9464a7e3e5" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.27:8080: connect: connection refused"
Oct 30 22:14:06.860965 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:14:06.860939    6133 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7b6759dd5d/isvc-logger-raw-f8e55-predictor-845877cbb-jmr69" podUID="6a44df26-dc83-4b1b-94ba-fd9464a7e3e5" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 22:14:06.882723 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:14:06.882697    6133 log.go:25] "Finished parsing log file" path="/var/log/pods/e2e-gw1-2e5a75ca98_isvc-secondary-795342-predictor-58fc4fd76b-lpb6n_c5487b74-692c-419b-8d07-c7074b6cdb7e/storage-initializer/0.log"
Oct 30 22:14:06.883196 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:14:06.882790    6133 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-2e5a75ca98/isvc-secondary-795342-predictor-58fc4fd76b-lpb6n" event={"ID":"c5487b74-692c-419b-8d07-c7074b6cdb7e","Type":"ContainerStarted","Data":"c41725a6b725e9ddc834efac25dc0f25a60d340da06dc2b5c4cb58f758f3666d"}
Oct 30 22:14:09.527307 ip-10-0-143-246 kubenswrapper[6133]: E1030 22:14:09.527262    6133 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6jppj" containerName="extractor"
Oct 30 22:14:09.894587 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:14:09.894505    6133 log.go:25] "Finished parsing log file" path="/var/log/pods/e2e-gw1-2e5a75ca98_isvc-secondary-795342-predictor-58fc4fd76b-lpb6n_c5487b74-692c-419b-8d07-c7074b6cdb7e/storage-initializer/1.log"
Oct 30 22:14:09.894898 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:14:09.894884    6133 log.go:25] "Finished parsing log file" path="/var/log/pods/e2e-gw1-2e5a75ca98_isvc-secondary-795342-predictor-58fc4fd76b-lpb6n_c5487b74-692c-419b-8d07-c7074b6cdb7e/storage-initializer/0.log"
Oct 30 22:14:09.894950 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:14:09.894915    6133 generic.go:356] "Generic (PLEG): container finished" podID="c5487b74-692c-419b-8d07-c7074b6cdb7e" containerID="c41725a6b725e9ddc834efac25dc0f25a60d340da06dc2b5c4cb58f758f3666d" exitCode=1
Oct 30 22:14:09.895006 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:14:09.894970    6133 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-2e5a75ca98/isvc-secondary-795342-predictor-58fc4fd76b-lpb6n" event={"ID":"c5487b74-692c-419b-8d07-c7074b6cdb7e","Type":"ContainerDied","Data":"c41725a6b725e9ddc834efac25dc0f25a60d340da06dc2b5c4cb58f758f3666d"}
Oct 30 22:14:09.895062 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:14:09.895039    6133 scope.go:122] "RemoveContainer" containerID="86d79169e8f01efff54d0af7dd5bbf98f7226f939f33dd3608ba619925d72da6"
Oct 30 22:14:09.895438 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:14:09.895412    6133 scope.go:122] "RemoveContainer" containerID="86d79169e8f01efff54d0af7dd5bbf98f7226f939f33dd3608ba619925d72da6"
Oct 30 22:14:09.905957 ip-10-0-143-246 kubenswrapper[6133]: E1030 22:14:09.905931    6133 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"storage-initializer\" with CrashLoopBackOff: \"back-off 10s restarting failed container=storage-initializer pod=isvc-secondary-795342-predictor-58fc4fd76b-lpb6n_e2e-gw1-2e5a75ca98(c5487b74-692c-419b-8d07-c7074b6cdb7e)\"" pod="e2e-gw1-2e5a75ca98/isvc-secondary-795342-predictor-58fc4fd76b-lpb6n" podUID="c5487b74-692c-419b-8d07-c7074b6cdb7e"
Oct 30 22:14:09.906131 ip-10-0-143-246 kubenswrapper[6133]: E1030 22:14:09.906104    6133 log.go:32] "RemoveContainer from runtime service failed" err="rpc error: code = Unknown desc = failed to delete container k8s_storage-initializer_isvc-secondary-795342-predictor-58fc4fd76b-lpb6n_e2e-gw1-2e5a75ca98_c5487b74-692c-419b-8d07-c7074b6cdb7e_0 in pod sandbox 39492b404f5e05be88106016a9dcff7d1afda73fcabdfa45505572c75db4b52e from index: no such id: '86d79169e8f01efff54d0af7dd5bbf98f7226f939f33dd3608ba619925d72da6'" containerID="86d79169e8f01efff54d0af7dd5bbf98f7226f939f33dd3608ba619925d72da6"
Oct 30 22:14:09.906184 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:14:09.906143    6133 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"86d79169e8f01efff54d0af7dd5bbf98f7226f939f33dd3608ba619925d72da6"} err="rpc error: code = Unknown desc = failed to delete container k8s_storage-initializer_isvc-secondary-795342-predictor-58fc4fd76b-lpb6n_e2e-gw1-2e5a75ca98_c5487b74-692c-419b-8d07-c7074b6cdb7e_0 in pod sandbox 39492b404f5e05be88106016a9dcff7d1afda73fcabdfa45505572c75db4b52e from index: no such id: '86d79169e8f01efff54d0af7dd5bbf98f7226f939f33dd3608ba619925d72da6'"
Oct 30 22:14:10.900637 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:14:10.900604    6133 log.go:25] "Finished parsing log file" path="/var/log/pods/e2e-gw1-2e5a75ca98_isvc-secondary-795342-predictor-58fc4fd76b-lpb6n_c5487b74-692c-419b-8d07-c7074b6cdb7e/storage-initializer/1.log"
Oct 30 22:14:16.861402 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:14:16.861355    6133 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7b6759dd5d/isvc-logger-raw-f8e55-predictor-845877cbb-jmr69" podUID="6a44df26-dc83-4b1b-94ba-fd9464a7e3e5" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.27:8080: connect: connection refused"
Oct 30 22:14:16.863686 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:14:16.861858    6133 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7b6759dd5d/isvc-logger-raw-f8e55-predictor-845877cbb-jmr69" podUID="6a44df26-dc83-4b1b-94ba-fd9464a7e3e5" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 22:14:18.534239 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:14:18.534207    6133 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw1-2e5a75ca98/isvc-secondary-795342-predictor-58fc4fd76b-lpb6n"]
Oct 30 22:14:18.663922 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:14:18.663897    6133 log.go:25] "Finished parsing log file" path="/var/log/pods/e2e-gw1-2e5a75ca98_isvc-secondary-795342-predictor-58fc4fd76b-lpb6n_c5487b74-692c-419b-8d07-c7074b6cdb7e/storage-initializer/1.log"
Oct 30 22:14:18.666173 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:14:18.666156    6133 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-2e5a75ca98/isvc-secondary-795342-predictor-58fc4fd76b-lpb6n"
Oct 30 22:14:18.788044 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:14:18.787941    6133 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/c5487b74-692c-419b-8d07-c7074b6cdb7e-proxy-tls\" (UniqueName: \"kubernetes.io/secret/c5487b74-692c-419b-8d07-c7074b6cdb7e-proxy-tls\") pod \"c5487b74-692c-419b-8d07-c7074b6cdb7e\" (UID: \"c5487b74-692c-419b-8d07-c7074b6cdb7e\") "
Oct 30 22:14:18.788044 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:14:18.787979    6133 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c5487b74-692c-419b-8d07-c7074b6cdb7e-cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/c5487b74-692c-419b-8d07-c7074b6cdb7e-cabundle-cert\") pod \"c5487b74-692c-419b-8d07-c7074b6cdb7e\" (UID: \"c5487b74-692c-419b-8d07-c7074b6cdb7e\") "
Oct 30 22:14:18.788278 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:14:18.788067    6133 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c5487b74-692c-419b-8d07-c7074b6cdb7e-kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/c5487b74-692c-419b-8d07-c7074b6cdb7e-kube-rbac-proxy-sar-config\") pod \"c5487b74-692c-419b-8d07-c7074b6cdb7e\" (UID: \"c5487b74-692c-419b-8d07-c7074b6cdb7e\") "
Oct 30 22:14:18.788278 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:14:18.788104    6133 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c5487b74-692c-419b-8d07-c7074b6cdb7e-kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/c5487b74-692c-419b-8d07-c7074b6cdb7e-kserve-provision-location\") pod \"c5487b74-692c-419b-8d07-c7074b6cdb7e\" (UID: \"c5487b74-692c-419b-8d07-c7074b6cdb7e\") "
Oct 30 22:14:18.788278 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:14:18.788177    6133 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c5487b74-692c-419b-8d07-c7074b6cdb7e-kube-api-access-qf7bd\" (UniqueName: \"kubernetes.io/projected/c5487b74-692c-419b-8d07-c7074b6cdb7e-kube-api-access-qf7bd\") pod \"c5487b74-692c-419b-8d07-c7074b6cdb7e\" (UID: \"c5487b74-692c-419b-8d07-c7074b6cdb7e\") "
Oct 30 22:14:18.788440 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:14:18.788405    6133 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c5487b74-692c-419b-8d07-c7074b6cdb7e-kserve-provision-location" pod "c5487b74-692c-419b-8d07-c7074b6cdb7e" (UID: "c5487b74-692c-419b-8d07-c7074b6cdb7e"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:14:18.788497 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:14:18.788441    6133 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c5487b74-692c-419b-8d07-c7074b6cdb7e-cabundle-cert" pod "c5487b74-692c-419b-8d07-c7074b6cdb7e" (UID: "c5487b74-692c-419b-8d07-c7074b6cdb7e"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:14:18.788544 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:14:18.788520    6133 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c5487b74-692c-419b-8d07-c7074b6cdb7e-kube-rbac-proxy-sar-config" pod "c5487b74-692c-419b-8d07-c7074b6cdb7e" (UID: "c5487b74-692c-419b-8d07-c7074b6cdb7e"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:14:18.790213 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:14:18.790191    6133 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c5487b74-692c-419b-8d07-c7074b6cdb7e-proxy-tls" pod "c5487b74-692c-419b-8d07-c7074b6cdb7e" (UID: "c5487b74-692c-419b-8d07-c7074b6cdb7e"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:14:18.790213 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:14:18.790194    6133 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c5487b74-692c-419b-8d07-c7074b6cdb7e-kube-api-access-qf7bd" pod "c5487b74-692c-419b-8d07-c7074b6cdb7e" (UID: "c5487b74-692c-419b-8d07-c7074b6cdb7e"). InnerVolumeSpecName "kube-api-access-qf7bd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:14:18.889653 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:14:18.889617    6133 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qf7bd\" (UniqueName: \"kubernetes.io/projected/c5487b74-692c-419b-8d07-c7074b6cdb7e-kube-api-access-qf7bd\") on node \"ip-10-0-143-246.ec2.internal\" DevicePath \"\""
Oct 30 22:14:18.889838 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:14:18.889668    6133 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/c5487b74-692c-419b-8d07-c7074b6cdb7e-proxy-tls\") on node \"ip-10-0-143-246.ec2.internal\" DevicePath \"\""
Oct 30 22:14:18.889838 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:14:18.889680    6133 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/c5487b74-692c-419b-8d07-c7074b6cdb7e-cabundle-cert\") on node \"ip-10-0-143-246.ec2.internal\" DevicePath \"\""
Oct 30 22:14:18.889838 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:14:18.889688    6133 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/c5487b74-692c-419b-8d07-c7074b6cdb7e-kube-rbac-proxy-sar-config\") on node \"ip-10-0-143-246.ec2.internal\" DevicePath \"\""
Oct 30 22:14:18.889838 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:14:18.889697    6133 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/c5487b74-692c-419b-8d07-c7074b6cdb7e-kserve-provision-location\") on node \"ip-10-0-143-246.ec2.internal\" DevicePath \"\""
Oct 30 22:14:18.930762 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:14:18.930737    6133 log.go:25] "Finished parsing log file" path="/var/log/pods/e2e-gw1-2e5a75ca98_isvc-secondary-795342-predictor-58fc4fd76b-lpb6n_c5487b74-692c-419b-8d07-c7074b6cdb7e/storage-initializer/1.log"
Oct 30 22:14:18.930919 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:14:18.930833    6133 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-2e5a75ca98/isvc-secondary-795342-predictor-58fc4fd76b-lpb6n"
Oct 30 22:14:18.930919 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:14:18.930836    6133 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-2e5a75ca98/isvc-secondary-795342-predictor-58fc4fd76b-lpb6n" event={"ID":"c5487b74-692c-419b-8d07-c7074b6cdb7e","Type":"ContainerDied","Data":"39492b404f5e05be88106016a9dcff7d1afda73fcabdfa45505572c75db4b52e"}
Oct 30 22:14:18.930919 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:14:18.930892    6133 scope.go:122] "RemoveContainer" containerID="c41725a6b725e9ddc834efac25dc0f25a60d340da06dc2b5c4cb58f758f3666d"
Oct 30 22:14:19.014432 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:14:19.014396    6133 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw1-2e5a75ca98/isvc-secondary-795342-predictor-58fc4fd76b-lpb6n"]
Oct 30 22:14:19.019401 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:14:19.019374    6133 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-gw1-2e5a75ca98/isvc-secondary-795342-predictor-58fc4fd76b-lpb6n"]
Oct 30 22:14:19.354790 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:14:19.354759    6133 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c5487b74-692c-419b-8d07-c7074b6cdb7e" path="/var/lib/kubelet/pods/c5487b74-692c-419b-8d07-c7074b6cdb7e/volumes"
Oct 30 22:14:26.860745 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:14:26.860700    6133 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7b6759dd5d/isvc-logger-raw-f8e55-predictor-845877cbb-jmr69" podUID="6a44df26-dc83-4b1b-94ba-fd9464a7e3e5" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.27:8080: connect: connection refused"
Oct 30 22:14:26.861299 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:14:26.861271    6133 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7b6759dd5d/isvc-logger-raw-f8e55-predictor-845877cbb-jmr69" podUID="6a44df26-dc83-4b1b-94ba-fd9464a7e3e5" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 22:14:36.860956 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:14:36.860861    6133 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7b6759dd5d/isvc-logger-raw-f8e55-predictor-845877cbb-jmr69" podUID="6a44df26-dc83-4b1b-94ba-fd9464a7e3e5" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.27:8080: connect: connection refused"
Oct 30 22:14:36.861518 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:14:36.861409    6133 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7b6759dd5d/isvc-logger-raw-f8e55-predictor-845877cbb-jmr69" podUID="6a44df26-dc83-4b1b-94ba-fd9464a7e3e5" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 22:14:39.528422 ip-10-0-143-246 kubenswrapper[6133]: E1030 22:14:39.528376    6133 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6jppj" containerName="extractor"
Oct 30 22:14:46.861202 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:14:46.861162    6133 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7b6759dd5d/isvc-logger-raw-f8e55-predictor-845877cbb-jmr69" podUID="6a44df26-dc83-4b1b-94ba-fd9464a7e3e5" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.27:8080: connect: connection refused"
Oct 30 22:14:46.861798 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:14:46.861618    6133 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7b6759dd5d/isvc-logger-raw-f8e55-predictor-845877cbb-jmr69" podUID="6a44df26-dc83-4b1b-94ba-fd9464a7e3e5" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 22:14:56.861388 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:14:56.861344    6133 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7b6759dd5d/isvc-logger-raw-f8e55-predictor-845877cbb-jmr69" podUID="6a44df26-dc83-4b1b-94ba-fd9464a7e3e5" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.27:8080: connect: connection refused"
Oct 30 22:14:56.862000 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:14:56.861965    6133 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7b6759dd5d/isvc-logger-raw-f8e55-predictor-845877cbb-jmr69" podUID="6a44df26-dc83-4b1b-94ba-fd9464a7e3e5" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 22:15:06.861145 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:15:06.861105    6133 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7b6759dd5d/isvc-logger-raw-f8e55-predictor-845877cbb-jmr69" podUID="6a44df26-dc83-4b1b-94ba-fd9464a7e3e5" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.27:8080: connect: connection refused"
Oct 30 22:15:06.861733 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:15:06.861615    6133 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7b6759dd5d/isvc-logger-raw-f8e55-predictor-845877cbb-jmr69" podUID="6a44df26-dc83-4b1b-94ba-fd9464a7e3e5" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 22:15:09.527681 ip-10-0-143-246 kubenswrapper[6133]: E1030 22:15:09.527645    6133 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6jppj" containerName="extractor"
Oct 30 22:15:16.861543 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:15:16.861496    6133 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7b6759dd5d/isvc-logger-raw-f8e55-predictor-845877cbb-jmr69" podUID="6a44df26-dc83-4b1b-94ba-fd9464a7e3e5" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.27:8080: connect: connection refused"
Oct 30 22:15:16.862070 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:15:16.861910    6133 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7b6759dd5d/isvc-logger-raw-f8e55-predictor-845877cbb-jmr69" podUID="6a44df26-dc83-4b1b-94ba-fd9464a7e3e5" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 22:15:18.351519 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:15:18.351478    6133 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7b6759dd5d/isvc-logger-raw-f8e55-predictor-845877cbb-jmr69" podUID="6a44df26-dc83-4b1b-94ba-fd9464a7e3e5" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.27:8080: connect: connection refused"
Oct 30 22:15:18.351934 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:15:18.351803    6133 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7b6759dd5d/isvc-logger-raw-f8e55-predictor-845877cbb-jmr69" podUID="6a44df26-dc83-4b1b-94ba-fd9464a7e3e5" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 22:15:28.352819 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:15:28.352781    6133 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw0-7b6759dd5d/isvc-logger-raw-f8e55-predictor-845877cbb-jmr69"
Oct 30 22:15:28.353314 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:15:28.352925    6133 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw0-7b6759dd5d/isvc-logger-raw-f8e55-predictor-845877cbb-jmr69"
Oct 30 22:15:39.526971 ip-10-0-143-246 kubenswrapper[6133]: E1030 22:15:39.526931    6133 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6jppj" containerName="extractor"
Oct 30 22:15:40.535287 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:15:40.535256    6133 log.go:25] "Finished parsing log file" path="/var/log/pods/e2e-gw0-7b6759dd5d_message-dumper-raw-f8e55-predictor-7bbf7d9766-68n76_be61d92f-8af7-40cd-82f4-3d4354ad7e1a/kserve-container/0.log"
Oct 30 22:15:41.130408 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:15:41.130375    6133 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw0-7b6759dd5d/isvc-logger-raw-f8e55-predictor-845877cbb-jmr69"]
Oct 30 22:15:41.130671 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:15:41.130642    6133 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw0-7b6759dd5d/isvc-logger-raw-f8e55-predictor-845877cbb-jmr69" podUID="6a44df26-dc83-4b1b-94ba-fd9464a7e3e5" containerName="kserve-container" containerID="cri-o://cb638c5b34fff17250c4daeda2386cdde72b6551c69a2235aeeef9dce795664b" gracePeriod=30
Oct 30 22:15:41.130778 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:15:41.130690    6133 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw0-7b6759dd5d/isvc-logger-raw-f8e55-predictor-845877cbb-jmr69" podUID="6a44df26-dc83-4b1b-94ba-fd9464a7e3e5" containerName="agent" containerID="cri-o://56c96fb5c2db55e02a683ed5aafdc56fa4d8a9116f3f2f63db25bdb7d2c3eccc" gracePeriod=30
Oct 30 22:15:41.130778 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:15:41.130691    6133 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw0-7b6759dd5d/isvc-logger-raw-f8e55-predictor-845877cbb-jmr69" podUID="6a44df26-dc83-4b1b-94ba-fd9464a7e3e5" containerName="kube-rbac-proxy" containerID="cri-o://502db0bdd6da1dc299bea2bc3a46b3dd59af9954385e38292a4f09e82666966c" gracePeriod=30
Oct 30 22:15:41.224999 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:15:41.224952    6133 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw0-7b6759dd5d/message-dumper-raw-f8e55-predictor-7bbf7d9766-68n76"]
Oct 30 22:15:41.225209 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:15:41.225187    6133 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw0-7b6759dd5d/message-dumper-raw-f8e55-predictor-7bbf7d9766-68n76" podUID="be61d92f-8af7-40cd-82f4-3d4354ad7e1a" containerName="kserve-container" containerID="cri-o://1d077c531610c47a816b61c3e782e2c29b42bcb93a730ad8f35a481532207566" gracePeriod=30
Oct 30 22:15:41.225279 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:15:41.225235    6133 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw0-7b6759dd5d/message-dumper-raw-f8e55-predictor-7bbf7d9766-68n76" podUID="be61d92f-8af7-40cd-82f4-3d4354ad7e1a" containerName="kube-rbac-proxy" containerID="cri-o://c9321c5d3e0b75c5d28faec941969912ef79a7585bbc96cc9c13bac013af0085" gracePeriod=30
Oct 30 22:15:41.261241 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:15:41.261201    6133 generic.go:356] "Generic (PLEG): container finished" podID="6a44df26-dc83-4b1b-94ba-fd9464a7e3e5" containerID="502db0bdd6da1dc299bea2bc3a46b3dd59af9954385e38292a4f09e82666966c" exitCode=0
Oct 30 22:15:41.261400 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:15:41.261273    6133 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-7b6759dd5d/isvc-logger-raw-f8e55-predictor-845877cbb-jmr69" event={"ID":"6a44df26-dc83-4b1b-94ba-fd9464a7e3e5","Type":"ContainerDied","Data":"502db0bdd6da1dc299bea2bc3a46b3dd59af9954385e38292a4f09e82666966c"}
Oct 30 22:15:41.465413 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:15:41.465385    6133 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-7b6759dd5d/message-dumper-raw-f8e55-predictor-7bbf7d9766-68n76"
Oct 30 22:15:41.587403 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:15:41.587366    6133 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/be61d92f-8af7-40cd-82f4-3d4354ad7e1a-kube-api-access-9cvf6\" (UniqueName: \"kubernetes.io/projected/be61d92f-8af7-40cd-82f4-3d4354ad7e1a-kube-api-access-9cvf6\") pod \"be61d92f-8af7-40cd-82f4-3d4354ad7e1a\" (UID: \"be61d92f-8af7-40cd-82f4-3d4354ad7e1a\") "
Oct 30 22:15:41.587869 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:15:41.587434    6133 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/be61d92f-8af7-40cd-82f4-3d4354ad7e1a-kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/be61d92f-8af7-40cd-82f4-3d4354ad7e1a-kube-rbac-proxy-sar-config\") pod \"be61d92f-8af7-40cd-82f4-3d4354ad7e1a\" (UID: \"be61d92f-8af7-40cd-82f4-3d4354ad7e1a\") "
Oct 30 22:15:41.587869 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:15:41.587457    6133 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/be61d92f-8af7-40cd-82f4-3d4354ad7e1a-proxy-tls\" (UniqueName: \"kubernetes.io/secret/be61d92f-8af7-40cd-82f4-3d4354ad7e1a-proxy-tls\") pod \"be61d92f-8af7-40cd-82f4-3d4354ad7e1a\" (UID: \"be61d92f-8af7-40cd-82f4-3d4354ad7e1a\") "
Oct 30 22:15:41.587869 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:15:41.587788    6133 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/be61d92f-8af7-40cd-82f4-3d4354ad7e1a-kube-rbac-proxy-sar-config" pod "be61d92f-8af7-40cd-82f4-3d4354ad7e1a" (UID: "be61d92f-8af7-40cd-82f4-3d4354ad7e1a"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:15:41.589568 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:15:41.589544    6133 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/be61d92f-8af7-40cd-82f4-3d4354ad7e1a-kube-api-access-9cvf6" pod "be61d92f-8af7-40cd-82f4-3d4354ad7e1a" (UID: "be61d92f-8af7-40cd-82f4-3d4354ad7e1a"). InnerVolumeSpecName "kube-api-access-9cvf6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:15:41.589641 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:15:41.589622    6133 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/be61d92f-8af7-40cd-82f4-3d4354ad7e1a-proxy-tls" pod "be61d92f-8af7-40cd-82f4-3d4354ad7e1a" (UID: "be61d92f-8af7-40cd-82f4-3d4354ad7e1a"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:15:41.688055 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:15:41.687929    6133 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9cvf6\" (UniqueName: \"kubernetes.io/projected/be61d92f-8af7-40cd-82f4-3d4354ad7e1a-kube-api-access-9cvf6\") on node \"ip-10-0-143-246.ec2.internal\" DevicePath \"\""
Oct 30 22:15:41.688055 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:15:41.687978    6133 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/be61d92f-8af7-40cd-82f4-3d4354ad7e1a-kube-rbac-proxy-sar-config\") on node \"ip-10-0-143-246.ec2.internal\" DevicePath \"\""
Oct 30 22:15:41.688055 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:15:41.688016    6133 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/be61d92f-8af7-40cd-82f4-3d4354ad7e1a-proxy-tls\") on node \"ip-10-0-143-246.ec2.internal\" DevicePath \"\""
Oct 30 22:15:41.855689 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:15:41.855645    6133 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7b6759dd5d/isvc-logger-raw-f8e55-predictor-845877cbb-jmr69" podUID="6a44df26-dc83-4b1b-94ba-fd9464a7e3e5" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.27:8643/healthz\": dial tcp 10.134.0.27:8643: connect: connection refused"
Oct 30 22:15:42.266096 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:15:42.266060    6133 generic.go:356] "Generic (PLEG): container finished" podID="be61d92f-8af7-40cd-82f4-3d4354ad7e1a" containerID="c9321c5d3e0b75c5d28faec941969912ef79a7585bbc96cc9c13bac013af0085" exitCode=0
Oct 30 22:15:42.266096 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:15:42.266085    6133 generic.go:356] "Generic (PLEG): container finished" podID="be61d92f-8af7-40cd-82f4-3d4354ad7e1a" containerID="1d077c531610c47a816b61c3e782e2c29b42bcb93a730ad8f35a481532207566" exitCode=2
Oct 30 22:15:42.266347 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:15:42.266112    6133 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-7b6759dd5d/message-dumper-raw-f8e55-predictor-7bbf7d9766-68n76"
Oct 30 22:15:42.266347 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:15:42.266125    6133 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-7b6759dd5d/message-dumper-raw-f8e55-predictor-7bbf7d9766-68n76" event={"ID":"be61d92f-8af7-40cd-82f4-3d4354ad7e1a","Type":"ContainerDied","Data":"c9321c5d3e0b75c5d28faec941969912ef79a7585bbc96cc9c13bac013af0085"}
Oct 30 22:15:42.266347 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:15:42.266172    6133 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-7b6759dd5d/message-dumper-raw-f8e55-predictor-7bbf7d9766-68n76" event={"ID":"be61d92f-8af7-40cd-82f4-3d4354ad7e1a","Type":"ContainerDied","Data":"1d077c531610c47a816b61c3e782e2c29b42bcb93a730ad8f35a481532207566"}
Oct 30 22:15:42.266347 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:15:42.266196    6133 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-7b6759dd5d/message-dumper-raw-f8e55-predictor-7bbf7d9766-68n76" event={"ID":"be61d92f-8af7-40cd-82f4-3d4354ad7e1a","Type":"ContainerDied","Data":"6722077e1c6fb03811bbc3bf783935f8e3b80489ecf11769afe7cc15f2930a9f"}
Oct 30 22:15:42.266347 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:15:42.266250    6133 scope.go:122] "RemoveContainer" containerID="c9321c5d3e0b75c5d28faec941969912ef79a7585bbc96cc9c13bac013af0085"
Oct 30 22:15:42.275325 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:15:42.275306    6133 scope.go:122] "RemoveContainer" containerID="1d077c531610c47a816b61c3e782e2c29b42bcb93a730ad8f35a481532207566"
Oct 30 22:15:42.283323 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:15:42.283295    6133 scope.go:122] "RemoveContainer" containerID="c9321c5d3e0b75c5d28faec941969912ef79a7585bbc96cc9c13bac013af0085"
Oct 30 22:15:42.283609 ip-10-0-143-246 kubenswrapper[6133]: E1030 22:15:42.283590    6133 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c9321c5d3e0b75c5d28faec941969912ef79a7585bbc96cc9c13bac013af0085\": container with ID starting with c9321c5d3e0b75c5d28faec941969912ef79a7585bbc96cc9c13bac013af0085 not found: ID does not exist" containerID="c9321c5d3e0b75c5d28faec941969912ef79a7585bbc96cc9c13bac013af0085"
Oct 30 22:15:42.283664 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:15:42.283622    6133 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c9321c5d3e0b75c5d28faec941969912ef79a7585bbc96cc9c13bac013af0085"} err="failed to get container status \"c9321c5d3e0b75c5d28faec941969912ef79a7585bbc96cc9c13bac013af0085\": rpc error: code = NotFound desc = could not find container \"c9321c5d3e0b75c5d28faec941969912ef79a7585bbc96cc9c13bac013af0085\": container with ID starting with c9321c5d3e0b75c5d28faec941969912ef79a7585bbc96cc9c13bac013af0085 not found: ID does not exist"
Oct 30 22:15:42.283664 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:15:42.283638    6133 scope.go:122] "RemoveContainer" containerID="1d077c531610c47a816b61c3e782e2c29b42bcb93a730ad8f35a481532207566"
Oct 30 22:15:42.283854 ip-10-0-143-246 kubenswrapper[6133]: E1030 22:15:42.283834    6133 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1d077c531610c47a816b61c3e782e2c29b42bcb93a730ad8f35a481532207566\": container with ID starting with 1d077c531610c47a816b61c3e782e2c29b42bcb93a730ad8f35a481532207566 not found: ID does not exist" containerID="1d077c531610c47a816b61c3e782e2c29b42bcb93a730ad8f35a481532207566"
Oct 30 22:15:42.283897 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:15:42.283860    6133 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1d077c531610c47a816b61c3e782e2c29b42bcb93a730ad8f35a481532207566"} err="failed to get container status \"1d077c531610c47a816b61c3e782e2c29b42bcb93a730ad8f35a481532207566\": rpc error: code = NotFound desc = could not find container \"1d077c531610c47a816b61c3e782e2c29b42bcb93a730ad8f35a481532207566\": container with ID starting with 1d077c531610c47a816b61c3e782e2c29b42bcb93a730ad8f35a481532207566 not found: ID does not exist"
Oct 30 22:15:42.283897 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:15:42.283872    6133 scope.go:122] "RemoveContainer" containerID="c9321c5d3e0b75c5d28faec941969912ef79a7585bbc96cc9c13bac013af0085"
Oct 30 22:15:42.284084 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:15:42.284065    6133 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c9321c5d3e0b75c5d28faec941969912ef79a7585bbc96cc9c13bac013af0085"} err="failed to get container status \"c9321c5d3e0b75c5d28faec941969912ef79a7585bbc96cc9c13bac013af0085\": rpc error: code = NotFound desc = could not find container \"c9321c5d3e0b75c5d28faec941969912ef79a7585bbc96cc9c13bac013af0085\": container with ID starting with c9321c5d3e0b75c5d28faec941969912ef79a7585bbc96cc9c13bac013af0085 not found: ID does not exist"
Oct 30 22:15:42.284126 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:15:42.284085    6133 scope.go:122] "RemoveContainer" containerID="1d077c531610c47a816b61c3e782e2c29b42bcb93a730ad8f35a481532207566"
Oct 30 22:15:42.284327 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:15:42.284310    6133 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1d077c531610c47a816b61c3e782e2c29b42bcb93a730ad8f35a481532207566"} err="failed to get container status \"1d077c531610c47a816b61c3e782e2c29b42bcb93a730ad8f35a481532207566\": rpc error: code = NotFound desc = could not find container \"1d077c531610c47a816b61c3e782e2c29b42bcb93a730ad8f35a481532207566\": container with ID starting with 1d077c531610c47a816b61c3e782e2c29b42bcb93a730ad8f35a481532207566 not found: ID does not exist"
Oct 30 22:15:42.320336 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:15:42.320305    6133 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw0-7b6759dd5d/message-dumper-raw-f8e55-predictor-7bbf7d9766-68n76"]
Oct 30 22:15:42.332863 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:15:42.332834    6133 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-gw0-7b6759dd5d/message-dumper-raw-f8e55-predictor-7bbf7d9766-68n76"]
Oct 30 22:15:43.355479 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:15:43.355443    6133 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="be61d92f-8af7-40cd-82f4-3d4354ad7e1a" path="/var/lib/kubelet/pods/be61d92f-8af7-40cd-82f4-3d4354ad7e1a/volumes"
Oct 30 22:15:46.855694 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:15:46.855640    6133 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7b6759dd5d/isvc-logger-raw-f8e55-predictor-845877cbb-jmr69" podUID="6a44df26-dc83-4b1b-94ba-fd9464a7e3e5" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.27:8643/healthz\": dial tcp 10.134.0.27:8643: connect: connection refused"
Oct 30 22:15:47.289544 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:15:47.289511    6133 generic.go:356] "Generic (PLEG): container finished" podID="6a44df26-dc83-4b1b-94ba-fd9464a7e3e5" containerID="cb638c5b34fff17250c4daeda2386cdde72b6551c69a2235aeeef9dce795664b" exitCode=0
Oct 30 22:15:47.289732 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:15:47.289584    6133 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-7b6759dd5d/isvc-logger-raw-f8e55-predictor-845877cbb-jmr69" event={"ID":"6a44df26-dc83-4b1b-94ba-fd9464a7e3e5","Type":"ContainerDied","Data":"cb638c5b34fff17250c4daeda2386cdde72b6551c69a2235aeeef9dce795664b"}
Oct 30 22:15:48.352201 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:15:48.352152    6133 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7b6759dd5d/isvc-logger-raw-f8e55-predictor-845877cbb-jmr69" podUID="6a44df26-dc83-4b1b-94ba-fd9464a7e3e5" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.27:8080: connect: connection refused"
Oct 30 22:15:48.352625 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:15:48.352478    6133 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7b6759dd5d/isvc-logger-raw-f8e55-predictor-845877cbb-jmr69" podUID="6a44df26-dc83-4b1b-94ba-fd9464a7e3e5" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 22:15:51.856136 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:15:51.856088    6133 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7b6759dd5d/isvc-logger-raw-f8e55-predictor-845877cbb-jmr69" podUID="6a44df26-dc83-4b1b-94ba-fd9464a7e3e5" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.27:8643/healthz\": dial tcp 10.134.0.27:8643: connect: connection refused"
Oct 30 22:15:51.856583 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:15:51.856248    6133 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw0-7b6759dd5d/isvc-logger-raw-f8e55-predictor-845877cbb-jmr69"
Oct 30 22:15:56.855475 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:15:56.855426    6133 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7b6759dd5d/isvc-logger-raw-f8e55-predictor-845877cbb-jmr69" podUID="6a44df26-dc83-4b1b-94ba-fd9464a7e3e5" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.27:8643/healthz\": dial tcp 10.134.0.27:8643: connect: connection refused"
Oct 30 22:15:58.351801 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:15:58.351744    6133 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7b6759dd5d/isvc-logger-raw-f8e55-predictor-845877cbb-jmr69" podUID="6a44df26-dc83-4b1b-94ba-fd9464a7e3e5" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.27:8080: connect: connection refused"
Oct 30 22:15:58.352297 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:15:58.352126    6133 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7b6759dd5d/isvc-logger-raw-f8e55-predictor-845877cbb-jmr69" podUID="6a44df26-dc83-4b1b-94ba-fd9464a7e3e5" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 22:16:01.855834 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:16:01.855792    6133 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7b6759dd5d/isvc-logger-raw-f8e55-predictor-845877cbb-jmr69" podUID="6a44df26-dc83-4b1b-94ba-fd9464a7e3e5" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.27:8643/healthz\": dial tcp 10.134.0.27:8643: connect: connection refused"
Oct 30 22:16:06.855518 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:16:06.855424    6133 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7b6759dd5d/isvc-logger-raw-f8e55-predictor-845877cbb-jmr69" podUID="6a44df26-dc83-4b1b-94ba-fd9464a7e3e5" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.27:8643/healthz\": dial tcp 10.134.0.27:8643: connect: connection refused"
Oct 30 22:16:08.351939 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:16:08.351899    6133 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7b6759dd5d/isvc-logger-raw-f8e55-predictor-845877cbb-jmr69" podUID="6a44df26-dc83-4b1b-94ba-fd9464a7e3e5" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.27:8080: connect: connection refused"
Oct 30 22:16:08.352482 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:16:08.352088    6133 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw0-7b6759dd5d/isvc-logger-raw-f8e55-predictor-845877cbb-jmr69"
Oct 30 22:16:08.352552 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:16:08.352481    6133 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7b6759dd5d/isvc-logger-raw-f8e55-predictor-845877cbb-jmr69" podUID="6a44df26-dc83-4b1b-94ba-fd9464a7e3e5" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 22:16:08.352623 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:16:08.352610    6133 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw0-7b6759dd5d/isvc-logger-raw-f8e55-predictor-845877cbb-jmr69"
Oct 30 22:16:09.527138 ip-10-0-143-246 kubenswrapper[6133]: E1030 22:16:09.527101    6133 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6jppj" containerName="extractor"
Oct 30 22:16:11.279965 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:16:11.279938    6133 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-7b6759dd5d/isvc-logger-raw-f8e55-predictor-845877cbb-jmr69"
Oct 30 22:16:11.322244 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:16:11.322206    6133 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/6a44df26-dc83-4b1b-94ba-fd9464a7e3e5-proxy-tls\" (UniqueName: \"kubernetes.io/secret/6a44df26-dc83-4b1b-94ba-fd9464a7e3e5-proxy-tls\") pod \"6a44df26-dc83-4b1b-94ba-fd9464a7e3e5\" (UID: \"6a44df26-dc83-4b1b-94ba-fd9464a7e3e5\") "
Oct 30 22:16:11.322423 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:16:11.322256    6133 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6a44df26-dc83-4b1b-94ba-fd9464a7e3e5-kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/6a44df26-dc83-4b1b-94ba-fd9464a7e3e5-kserve-provision-location\") pod \"6a44df26-dc83-4b1b-94ba-fd9464a7e3e5\" (UID: \"6a44df26-dc83-4b1b-94ba-fd9464a7e3e5\") "
Oct 30 22:16:11.322423 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:16:11.322312    6133 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/6a44df26-dc83-4b1b-94ba-fd9464a7e3e5-cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/6a44df26-dc83-4b1b-94ba-fd9464a7e3e5-cabundle-cert\") pod \"6a44df26-dc83-4b1b-94ba-fd9464a7e3e5\" (UID: \"6a44df26-dc83-4b1b-94ba-fd9464a7e3e5\") "
Oct 30 22:16:11.322423 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:16:11.322332    6133 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6a44df26-dc83-4b1b-94ba-fd9464a7e3e5-kube-api-access-csvbz\" (UniqueName: \"kubernetes.io/projected/6a44df26-dc83-4b1b-94ba-fd9464a7e3e5-kube-api-access-csvbz\") pod \"6a44df26-dc83-4b1b-94ba-fd9464a7e3e5\" (UID: \"6a44df26-dc83-4b1b-94ba-fd9464a7e3e5\") "
Oct 30 22:16:11.322423 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:16:11.322353    6133 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/6a44df26-dc83-4b1b-94ba-fd9464a7e3e5-kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/6a44df26-dc83-4b1b-94ba-fd9464a7e3e5-kube-rbac-proxy-sar-config\") pod \"6a44df26-dc83-4b1b-94ba-fd9464a7e3e5\" (UID: \"6a44df26-dc83-4b1b-94ba-fd9464a7e3e5\") "
Oct 30 22:16:11.322723 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:16:11.322691    6133 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6a44df26-dc83-4b1b-94ba-fd9464a7e3e5-kserve-provision-location" pod "6a44df26-dc83-4b1b-94ba-fd9464a7e3e5" (UID: "6a44df26-dc83-4b1b-94ba-fd9464a7e3e5"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:16:11.322792 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:16:11.322703    6133 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6a44df26-dc83-4b1b-94ba-fd9464a7e3e5-cabundle-cert" pod "6a44df26-dc83-4b1b-94ba-fd9464a7e3e5" (UID: "6a44df26-dc83-4b1b-94ba-fd9464a7e3e5"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:16:11.322792 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:16:11.322778    6133 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6a44df26-dc83-4b1b-94ba-fd9464a7e3e5-kube-rbac-proxy-sar-config" pod "6a44df26-dc83-4b1b-94ba-fd9464a7e3e5" (UID: "6a44df26-dc83-4b1b-94ba-fd9464a7e3e5"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:16:11.324464 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:16:11.324438    6133 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6a44df26-dc83-4b1b-94ba-fd9464a7e3e5-proxy-tls" pod "6a44df26-dc83-4b1b-94ba-fd9464a7e3e5" (UID: "6a44df26-dc83-4b1b-94ba-fd9464a7e3e5"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:16:11.324595 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:16:11.324483    6133 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6a44df26-dc83-4b1b-94ba-fd9464a7e3e5-kube-api-access-csvbz" pod "6a44df26-dc83-4b1b-94ba-fd9464a7e3e5" (UID: "6a44df26-dc83-4b1b-94ba-fd9464a7e3e5"). InnerVolumeSpecName "kube-api-access-csvbz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:16:11.375930 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:16:11.375897    6133 generic.go:356] "Generic (PLEG): container finished" podID="6a44df26-dc83-4b1b-94ba-fd9464a7e3e5" containerID="56c96fb5c2db55e02a683ed5aafdc56fa4d8a9116f3f2f63db25bdb7d2c3eccc" exitCode=0
Oct 30 22:16:11.376133 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:16:11.375968    6133 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-7b6759dd5d/isvc-logger-raw-f8e55-predictor-845877cbb-jmr69"
Oct 30 22:16:11.376133 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:16:11.375978    6133 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-7b6759dd5d/isvc-logger-raw-f8e55-predictor-845877cbb-jmr69" event={"ID":"6a44df26-dc83-4b1b-94ba-fd9464a7e3e5","Type":"ContainerDied","Data":"56c96fb5c2db55e02a683ed5aafdc56fa4d8a9116f3f2f63db25bdb7d2c3eccc"}
Oct 30 22:16:11.376133 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:16:11.376062    6133 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-7b6759dd5d/isvc-logger-raw-f8e55-predictor-845877cbb-jmr69" event={"ID":"6a44df26-dc83-4b1b-94ba-fd9464a7e3e5","Type":"ContainerDied","Data":"5c83c4edd76c3b893391ea04660ce0869a6c2bca5212e9af1ccebdc883063c47"}
Oct 30 22:16:11.376133 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:16:11.376092    6133 scope.go:122] "RemoveContainer" containerID="56c96fb5c2db55e02a683ed5aafdc56fa4d8a9116f3f2f63db25bdb7d2c3eccc"
Oct 30 22:16:11.384055 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:16:11.384033    6133 scope.go:122] "RemoveContainer" containerID="502db0bdd6da1dc299bea2bc3a46b3dd59af9954385e38292a4f09e82666966c"
Oct 30 22:16:11.391555 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:16:11.391536    6133 scope.go:122] "RemoveContainer" containerID="cb638c5b34fff17250c4daeda2386cdde72b6551c69a2235aeeef9dce795664b"
Oct 30 22:16:11.398921 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:16:11.398902    6133 scope.go:122] "RemoveContainer" containerID="8ecd485ecaab338dc63e19ea6dec0997fab3f47dc134975200508788fc3ba283"
Oct 30 22:16:11.406253 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:16:11.406234    6133 scope.go:122] "RemoveContainer" containerID="56c96fb5c2db55e02a683ed5aafdc56fa4d8a9116f3f2f63db25bdb7d2c3eccc"
Oct 30 22:16:11.406540 ip-10-0-143-246 kubenswrapper[6133]: E1030 22:16:11.406520    6133 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"56c96fb5c2db55e02a683ed5aafdc56fa4d8a9116f3f2f63db25bdb7d2c3eccc\": container with ID starting with 56c96fb5c2db55e02a683ed5aafdc56fa4d8a9116f3f2f63db25bdb7d2c3eccc not found: ID does not exist" containerID="56c96fb5c2db55e02a683ed5aafdc56fa4d8a9116f3f2f63db25bdb7d2c3eccc"
Oct 30 22:16:11.406595 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:16:11.406553    6133 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"56c96fb5c2db55e02a683ed5aafdc56fa4d8a9116f3f2f63db25bdb7d2c3eccc"} err="failed to get container status \"56c96fb5c2db55e02a683ed5aafdc56fa4d8a9116f3f2f63db25bdb7d2c3eccc\": rpc error: code = NotFound desc = could not find container \"56c96fb5c2db55e02a683ed5aafdc56fa4d8a9116f3f2f63db25bdb7d2c3eccc\": container with ID starting with 56c96fb5c2db55e02a683ed5aafdc56fa4d8a9116f3f2f63db25bdb7d2c3eccc not found: ID does not exist"
Oct 30 22:16:11.406595 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:16:11.406574    6133 scope.go:122] "RemoveContainer" containerID="502db0bdd6da1dc299bea2bc3a46b3dd59af9954385e38292a4f09e82666966c"
Oct 30 22:16:11.406833 ip-10-0-143-246 kubenswrapper[6133]: E1030 22:16:11.406816    6133 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"502db0bdd6da1dc299bea2bc3a46b3dd59af9954385e38292a4f09e82666966c\": container with ID starting with 502db0bdd6da1dc299bea2bc3a46b3dd59af9954385e38292a4f09e82666966c not found: ID does not exist" containerID="502db0bdd6da1dc299bea2bc3a46b3dd59af9954385e38292a4f09e82666966c"
Oct 30 22:16:11.406881 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:16:11.406840    6133 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"502db0bdd6da1dc299bea2bc3a46b3dd59af9954385e38292a4f09e82666966c"} err="failed to get container status \"502db0bdd6da1dc299bea2bc3a46b3dd59af9954385e38292a4f09e82666966c\": rpc error: code = NotFound desc = could not find container \"502db0bdd6da1dc299bea2bc3a46b3dd59af9954385e38292a4f09e82666966c\": container with ID starting with 502db0bdd6da1dc299bea2bc3a46b3dd59af9954385e38292a4f09e82666966c not found: ID does not exist"
Oct 30 22:16:11.406881 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:16:11.406855    6133 scope.go:122] "RemoveContainer" containerID="cb638c5b34fff17250c4daeda2386cdde72b6551c69a2235aeeef9dce795664b"
Oct 30 22:16:11.407113 ip-10-0-143-246 kubenswrapper[6133]: E1030 22:16:11.407095    6133 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cb638c5b34fff17250c4daeda2386cdde72b6551c69a2235aeeef9dce795664b\": container with ID starting with cb638c5b34fff17250c4daeda2386cdde72b6551c69a2235aeeef9dce795664b not found: ID does not exist" containerID="cb638c5b34fff17250c4daeda2386cdde72b6551c69a2235aeeef9dce795664b"
Oct 30 22:16:11.407190 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:16:11.407116    6133 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cb638c5b34fff17250c4daeda2386cdde72b6551c69a2235aeeef9dce795664b"} err="failed to get container status \"cb638c5b34fff17250c4daeda2386cdde72b6551c69a2235aeeef9dce795664b\": rpc error: code = NotFound desc = could not find container \"cb638c5b34fff17250c4daeda2386cdde72b6551c69a2235aeeef9dce795664b\": container with ID starting with cb638c5b34fff17250c4daeda2386cdde72b6551c69a2235aeeef9dce795664b not found: ID does not exist"
Oct 30 22:16:11.407190 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:16:11.407130    6133 scope.go:122] "RemoveContainer" containerID="8ecd485ecaab338dc63e19ea6dec0997fab3f47dc134975200508788fc3ba283"
Oct 30 22:16:11.407383 ip-10-0-143-246 kubenswrapper[6133]: E1030 22:16:11.407366    6133 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8ecd485ecaab338dc63e19ea6dec0997fab3f47dc134975200508788fc3ba283\": container with ID starting with 8ecd485ecaab338dc63e19ea6dec0997fab3f47dc134975200508788fc3ba283 not found: ID does not exist" containerID="8ecd485ecaab338dc63e19ea6dec0997fab3f47dc134975200508788fc3ba283"
Oct 30 22:16:11.407429 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:16:11.407388    6133 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8ecd485ecaab338dc63e19ea6dec0997fab3f47dc134975200508788fc3ba283"} err="failed to get container status \"8ecd485ecaab338dc63e19ea6dec0997fab3f47dc134975200508788fc3ba283\": rpc error: code = NotFound desc = could not find container \"8ecd485ecaab338dc63e19ea6dec0997fab3f47dc134975200508788fc3ba283\": container with ID starting with 8ecd485ecaab338dc63e19ea6dec0997fab3f47dc134975200508788fc3ba283 not found: ID does not exist"
Oct 30 22:16:11.423332 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:16:11.423308    6133 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/6a44df26-dc83-4b1b-94ba-fd9464a7e3e5-cabundle-cert\") on node \"ip-10-0-143-246.ec2.internal\" DevicePath \"\""
Oct 30 22:16:11.423332 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:16:11.423332    6133 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-csvbz\" (UniqueName: \"kubernetes.io/projected/6a44df26-dc83-4b1b-94ba-fd9464a7e3e5-kube-api-access-csvbz\") on node \"ip-10-0-143-246.ec2.internal\" DevicePath \"\""
Oct 30 22:16:11.423483 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:16:11.423342    6133 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/6a44df26-dc83-4b1b-94ba-fd9464a7e3e5-kube-rbac-proxy-sar-config\") on node \"ip-10-0-143-246.ec2.internal\" DevicePath \"\""
Oct 30 22:16:11.423483 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:16:11.423354    6133 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/6a44df26-dc83-4b1b-94ba-fd9464a7e3e5-proxy-tls\") on node \"ip-10-0-143-246.ec2.internal\" DevicePath \"\""
Oct 30 22:16:11.423483 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:16:11.423362    6133 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/6a44df26-dc83-4b1b-94ba-fd9464a7e3e5-kserve-provision-location\") on node \"ip-10-0-143-246.ec2.internal\" DevicePath \"\""
Oct 30 22:16:11.430765 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:16:11.430739    6133 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw0-7b6759dd5d/isvc-logger-raw-f8e55-predictor-845877cbb-jmr69"]
Oct 30 22:16:11.436491 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:16:11.436459    6133 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-gw0-7b6759dd5d/isvc-logger-raw-f8e55-predictor-845877cbb-jmr69"]
Oct 30 22:16:13.134869 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:16:13.134834    6133 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-gw0-7b6759dd5d/isvc-sklearn-scale-raw-9167b-predictor-6c577cd946-458qh"]
Oct 30 22:16:13.135368 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:16:13.135145    6133 cpu_manager.go:414] "containerMap: removing container" podUID="6a44df26-dc83-4b1b-94ba-fd9464a7e3e5" containerName="kserve-container"
Oct 30 22:16:13.135368 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:16:13.135157    6133 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6a44df26-dc83-4b1b-94ba-fd9464a7e3e5" containerName="kserve-container"
Oct 30 22:16:13.135368 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:16:13.135169    6133 cpu_manager.go:414] "containerMap: removing container" podUID="c5487b74-692c-419b-8d07-c7074b6cdb7e" containerName="storage-initializer"
Oct 30 22:16:13.135368 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:16:13.135174    6133 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c5487b74-692c-419b-8d07-c7074b6cdb7e" containerName="storage-initializer"
Oct 30 22:16:13.135368 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:16:13.135181    6133 cpu_manager.go:414] "containerMap: removing container" podUID="be61d92f-8af7-40cd-82f4-3d4354ad7e1a" containerName="kserve-container"
Oct 30 22:16:13.135368 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:16:13.135186    6133 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="be61d92f-8af7-40cd-82f4-3d4354ad7e1a" containerName="kserve-container"
Oct 30 22:16:13.135368 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:16:13.135196    6133 cpu_manager.go:414] "containerMap: removing container" podUID="6a44df26-dc83-4b1b-94ba-fd9464a7e3e5" containerName="kube-rbac-proxy"
Oct 30 22:16:13.135368 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:16:13.135201    6133 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6a44df26-dc83-4b1b-94ba-fd9464a7e3e5" containerName="kube-rbac-proxy"
Oct 30 22:16:13.135368 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:16:13.135211    6133 cpu_manager.go:414] "containerMap: removing container" podUID="be61d92f-8af7-40cd-82f4-3d4354ad7e1a" containerName="kube-rbac-proxy"
Oct 30 22:16:13.135368 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:16:13.135216    6133 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="be61d92f-8af7-40cd-82f4-3d4354ad7e1a" containerName="kube-rbac-proxy"
Oct 30 22:16:13.135368 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:16:13.135222    6133 cpu_manager.go:414] "containerMap: removing container" podUID="6a44df26-dc83-4b1b-94ba-fd9464a7e3e5" containerName="agent"
Oct 30 22:16:13.135368 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:16:13.135227    6133 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6a44df26-dc83-4b1b-94ba-fd9464a7e3e5" containerName="agent"
Oct 30 22:16:13.135368 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:16:13.135238    6133 cpu_manager.go:414] "containerMap: removing container" podUID="6a44df26-dc83-4b1b-94ba-fd9464a7e3e5" containerName="storage-initializer"
Oct 30 22:16:13.135368 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:16:13.135243    6133 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6a44df26-dc83-4b1b-94ba-fd9464a7e3e5" containerName="storage-initializer"
Oct 30 22:16:13.135368 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:16:13.135293    6133 memory_manager.go:365] "RemoveStaleState removing state" podUID="6a44df26-dc83-4b1b-94ba-fd9464a7e3e5" containerName="kserve-container"
Oct 30 22:16:13.135368 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:16:13.135300    6133 memory_manager.go:365] "RemoveStaleState removing state" podUID="c5487b74-692c-419b-8d07-c7074b6cdb7e" containerName="storage-initializer"
Oct 30 22:16:13.135368 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:16:13.135305    6133 memory_manager.go:365] "RemoveStaleState removing state" podUID="be61d92f-8af7-40cd-82f4-3d4354ad7e1a" containerName="kserve-container"
Oct 30 22:16:13.135368 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:16:13.135312    6133 memory_manager.go:365] "RemoveStaleState removing state" podUID="6a44df26-dc83-4b1b-94ba-fd9464a7e3e5" containerName="kube-rbac-proxy"
Oct 30 22:16:13.135368 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:16:13.135319    6133 memory_manager.go:365] "RemoveStaleState removing state" podUID="be61d92f-8af7-40cd-82f4-3d4354ad7e1a" containerName="kube-rbac-proxy"
Oct 30 22:16:13.135368 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:16:13.135324    6133 memory_manager.go:365] "RemoveStaleState removing state" podUID="6a44df26-dc83-4b1b-94ba-fd9464a7e3e5" containerName="agent"
Oct 30 22:16:13.135368 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:16:13.135330    6133 memory_manager.go:365] "RemoveStaleState removing state" podUID="c5487b74-692c-419b-8d07-c7074b6cdb7e" containerName="storage-initializer"
Oct 30 22:16:13.135368 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:16:13.135378    6133 cpu_manager.go:414] "containerMap: removing container" podUID="c5487b74-692c-419b-8d07-c7074b6cdb7e" containerName="storage-initializer"
Oct 30 22:16:13.136002 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:16:13.135385    6133 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c5487b74-692c-419b-8d07-c7074b6cdb7e" containerName="storage-initializer"
Oct 30 22:16:13.138394 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:16:13.138375    6133 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-7b6759dd5d/isvc-sklearn-scale-raw-9167b-predictor-6c577cd946-458qh"
Oct 30 22:16:13.168659 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:16:13.168628    6133 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw0-7b6759dd5d\"/\"seaweedfs-s3-creds\""
Oct 30 22:16:13.168848 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:16:13.168690    6133 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw0-7b6759dd5d\"/\"default-dockercfg-4b7xh\""
Oct 30 22:16:13.168848 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:16:13.168713    6133 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw0-7b6759dd5d\"/\"openshift-service-ca.crt\""
Oct 30 22:16:13.168848 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:16:13.168628    6133 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw0-7b6759dd5d\"/\"odh-kserve-custom-ca-bundle\""
Oct 30 22:16:13.168848 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:16:13.168628    6133 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw0-7b6759dd5d\"/\"isvc-sklearn-scale-raw-9167b-kube-rbac-proxy-sar-config\""
Oct 30 22:16:13.168848 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:16:13.168723    6133 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw0-7b6759dd5d\"/\"kube-root-ca.crt\""
Oct 30 22:16:13.168848 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:16:13.168722    6133 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw0-7b6759dd5d\"/\"isvc-sklearn-scale-raw-9167b-predictor-serving-cert\""
Oct 30 22:16:13.169638 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:16:13.169616    6133 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw0-7b6759dd5d/isvc-sklearn-scale-raw-9167b-predictor-6c577cd946-458qh"]
Oct 30 22:16:13.239175 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:16:13.239141    6133 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/acaaf46b-acde-4a9d-b95f-9081342e5009-proxy-tls\") pod \"isvc-sklearn-scale-raw-9167b-predictor-6c577cd946-458qh\" (UID: \"acaaf46b-acde-4a9d-b95f-9081342e5009\") " pod="e2e-gw0-7b6759dd5d/isvc-sklearn-scale-raw-9167b-predictor-6c577cd946-458qh"
Oct 30 22:16:13.239175 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:16:13.239178    6133 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/acaaf46b-acde-4a9d-b95f-9081342e5009-cabundle-cert\") pod \"isvc-sklearn-scale-raw-9167b-predictor-6c577cd946-458qh\" (UID: \"acaaf46b-acde-4a9d-b95f-9081342e5009\") " pod="e2e-gw0-7b6759dd5d/isvc-sklearn-scale-raw-9167b-predictor-6c577cd946-458qh"
Oct 30 22:16:13.239421 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:16:13.239210    6133 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lq79t\" (UniqueName: \"kubernetes.io/projected/acaaf46b-acde-4a9d-b95f-9081342e5009-kube-api-access-lq79t\") pod \"isvc-sklearn-scale-raw-9167b-predictor-6c577cd946-458qh\" (UID: \"acaaf46b-acde-4a9d-b95f-9081342e5009\") " pod="e2e-gw0-7b6759dd5d/isvc-sklearn-scale-raw-9167b-predictor-6c577cd946-458qh"
Oct 30 22:16:13.239421 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:16:13.239312    6133 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/acaaf46b-acde-4a9d-b95f-9081342e5009-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-scale-raw-9167b-predictor-6c577cd946-458qh\" (UID: \"acaaf46b-acde-4a9d-b95f-9081342e5009\") " pod="e2e-gw0-7b6759dd5d/isvc-sklearn-scale-raw-9167b-predictor-6c577cd946-458qh"
Oct 30 22:16:13.239421 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:16:13.239357    6133 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/acaaf46b-acde-4a9d-b95f-9081342e5009-kserve-provision-location\") pod \"isvc-sklearn-scale-raw-9167b-predictor-6c577cd946-458qh\" (UID: \"acaaf46b-acde-4a9d-b95f-9081342e5009\") " pod="e2e-gw0-7b6759dd5d/isvc-sklearn-scale-raw-9167b-predictor-6c577cd946-458qh"
Oct 30 22:16:13.340008 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:16:13.339957    6133 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/acaaf46b-acde-4a9d-b95f-9081342e5009-kserve-provision-location\") pod \"isvc-sklearn-scale-raw-9167b-predictor-6c577cd946-458qh\" (UID: \"acaaf46b-acde-4a9d-b95f-9081342e5009\") " pod="e2e-gw0-7b6759dd5d/isvc-sklearn-scale-raw-9167b-predictor-6c577cd946-458qh"
Oct 30 22:16:13.340194 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:16:13.340103    6133 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/acaaf46b-acde-4a9d-b95f-9081342e5009-proxy-tls\") pod \"isvc-sklearn-scale-raw-9167b-predictor-6c577cd946-458qh\" (UID: \"acaaf46b-acde-4a9d-b95f-9081342e5009\") " pod="e2e-gw0-7b6759dd5d/isvc-sklearn-scale-raw-9167b-predictor-6c577cd946-458qh"
Oct 30 22:16:13.340194 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:16:13.340127    6133 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/acaaf46b-acde-4a9d-b95f-9081342e5009-cabundle-cert\") pod \"isvc-sklearn-scale-raw-9167b-predictor-6c577cd946-458qh\" (UID: \"acaaf46b-acde-4a9d-b95f-9081342e5009\") " pod="e2e-gw0-7b6759dd5d/isvc-sklearn-scale-raw-9167b-predictor-6c577cd946-458qh"
Oct 30 22:16:13.340194 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:16:13.340149    6133 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lq79t\" (UniqueName: \"kubernetes.io/projected/acaaf46b-acde-4a9d-b95f-9081342e5009-kube-api-access-lq79t\") pod \"isvc-sklearn-scale-raw-9167b-predictor-6c577cd946-458qh\" (UID: \"acaaf46b-acde-4a9d-b95f-9081342e5009\") " pod="e2e-gw0-7b6759dd5d/isvc-sklearn-scale-raw-9167b-predictor-6c577cd946-458qh"
Oct 30 22:16:13.340194 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:16:13.340166    6133 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/acaaf46b-acde-4a9d-b95f-9081342e5009-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-scale-raw-9167b-predictor-6c577cd946-458qh\" (UID: \"acaaf46b-acde-4a9d-b95f-9081342e5009\") " pod="e2e-gw0-7b6759dd5d/isvc-sklearn-scale-raw-9167b-predictor-6c577cd946-458qh"
Oct 30 22:16:13.340512 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:16:13.340486    6133 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/acaaf46b-acde-4a9d-b95f-9081342e5009-kserve-provision-location\") pod \"isvc-sklearn-scale-raw-9167b-predictor-6c577cd946-458qh\" (UID: \"acaaf46b-acde-4a9d-b95f-9081342e5009\") " pod="e2e-gw0-7b6759dd5d/isvc-sklearn-scale-raw-9167b-predictor-6c577cd946-458qh"
Oct 30 22:16:13.340845 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:16:13.340819    6133 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/acaaf46b-acde-4a9d-b95f-9081342e5009-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-scale-raw-9167b-predictor-6c577cd946-458qh\" (UID: \"acaaf46b-acde-4a9d-b95f-9081342e5009\") " pod="e2e-gw0-7b6759dd5d/isvc-sklearn-scale-raw-9167b-predictor-6c577cd946-458qh"
Oct 30 22:16:13.340947 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:16:13.340848    6133 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/acaaf46b-acde-4a9d-b95f-9081342e5009-cabundle-cert\") pod \"isvc-sklearn-scale-raw-9167b-predictor-6c577cd946-458qh\" (UID: \"acaaf46b-acde-4a9d-b95f-9081342e5009\") " pod="e2e-gw0-7b6759dd5d/isvc-sklearn-scale-raw-9167b-predictor-6c577cd946-458qh"
Oct 30 22:16:13.342696 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:16:13.342674    6133 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/acaaf46b-acde-4a9d-b95f-9081342e5009-proxy-tls\") pod \"isvc-sklearn-scale-raw-9167b-predictor-6c577cd946-458qh\" (UID: \"acaaf46b-acde-4a9d-b95f-9081342e5009\") " pod="e2e-gw0-7b6759dd5d/isvc-sklearn-scale-raw-9167b-predictor-6c577cd946-458qh"
Oct 30 22:16:13.355338 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:16:13.355305    6133 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6a44df26-dc83-4b1b-94ba-fd9464a7e3e5" path="/var/lib/kubelet/pods/6a44df26-dc83-4b1b-94ba-fd9464a7e3e5/volumes"
Oct 30 22:16:13.361660 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:16:13.361633    6133 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lq79t\" (UniqueName: \"kubernetes.io/projected/acaaf46b-acde-4a9d-b95f-9081342e5009-kube-api-access-lq79t\") pod \"isvc-sklearn-scale-raw-9167b-predictor-6c577cd946-458qh\" (UID: \"acaaf46b-acde-4a9d-b95f-9081342e5009\") " pod="e2e-gw0-7b6759dd5d/isvc-sklearn-scale-raw-9167b-predictor-6c577cd946-458qh"
Oct 30 22:16:13.455347 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:16:13.455254    6133 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-7b6759dd5d/isvc-sklearn-scale-raw-9167b-predictor-6c577cd946-458qh"
Oct 30 22:16:13.761911 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:16:13.761883    6133 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw0-7b6759dd5d/isvc-sklearn-scale-raw-9167b-predictor-6c577cd946-458qh"]
Oct 30 22:16:13.762322 ip-10-0-143-246 kubenswrapper[6133]: W1030 22:16:13.762298    6133 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podacaaf46b_acde_4a9d_b95f_9081342e5009.slice/crio-14229531744c890faf537eab1fcb26c5ae5a23cad7e2fd30db99dc6cb8b2bec5 WatchSource:0}: Error finding container 14229531744c890faf537eab1fcb26c5ae5a23cad7e2fd30db99dc6cb8b2bec5: Status 404 returned error can't find the container with id 14229531744c890faf537eab1fcb26c5ae5a23cad7e2fd30db99dc6cb8b2bec5
Oct 30 22:16:14.390711 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:16:14.390670    6133 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-7b6759dd5d/isvc-sklearn-scale-raw-9167b-predictor-6c577cd946-458qh" event={"ID":"acaaf46b-acde-4a9d-b95f-9081342e5009","Type":"ContainerStarted","Data":"6e79f0dfed2e68c95d842f654ea69ba5eb378f6afe9ce7a4540670674c487600"}
Oct 30 22:16:14.391122 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:16:14.390743    6133 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-7b6759dd5d/isvc-sklearn-scale-raw-9167b-predictor-6c577cd946-458qh" event={"ID":"acaaf46b-acde-4a9d-b95f-9081342e5009","Type":"ContainerStarted","Data":"14229531744c890faf537eab1fcb26c5ae5a23cad7e2fd30db99dc6cb8b2bec5"}
Oct 30 22:16:18.406123 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:16:18.406085    6133 generic.go:356] "Generic (PLEG): container finished" podID="acaaf46b-acde-4a9d-b95f-9081342e5009" containerID="6e79f0dfed2e68c95d842f654ea69ba5eb378f6afe9ce7a4540670674c487600" exitCode=0
Oct 30 22:16:18.406532 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:16:18.406158    6133 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-7b6759dd5d/isvc-sklearn-scale-raw-9167b-predictor-6c577cd946-458qh" event={"ID":"acaaf46b-acde-4a9d-b95f-9081342e5009","Type":"ContainerDied","Data":"6e79f0dfed2e68c95d842f654ea69ba5eb378f6afe9ce7a4540670674c487600"}
Oct 30 22:16:19.411694 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:16:19.411659    6133 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-7b6759dd5d/isvc-sklearn-scale-raw-9167b-predictor-6c577cd946-458qh" event={"ID":"acaaf46b-acde-4a9d-b95f-9081342e5009","Type":"ContainerStarted","Data":"3150a3723af688a2f690829e7dde75140e5b41f1d10adfda24358c3156cb2746"}
Oct 30 22:16:19.412151 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:16:19.411722    6133 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-7b6759dd5d/isvc-sklearn-scale-raw-9167b-predictor-6c577cd946-458qh" event={"ID":"acaaf46b-acde-4a9d-b95f-9081342e5009","Type":"ContainerStarted","Data":"dc09e1be0cf893de26c652d4708a4c43e5ce6a250d68700bf29068d258984d26"}
Oct 30 22:16:19.412151 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:16:19.412014    6133 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw0-7b6759dd5d/isvc-sklearn-scale-raw-9167b-predictor-6c577cd946-458qh"
Oct 30 22:16:19.458654 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:16:19.458593    6133 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-gw0-7b6759dd5d/isvc-sklearn-scale-raw-9167b-predictor-6c577cd946-458qh" podStartSLOduration=2.521406422 podStartE2EDuration="6.458574546s" totalImagesPullingTime="0s" totalInitContainerRuntime="3.937168124s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:16:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:16:19.458460024 +0000 UTC m=+1286.219215915" watchObservedRunningTime="2026-10-30 22:16:19.458574546 +0000 UTC m=+1286.219330436"
Oct 30 22:16:20.416462 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:16:20.416429    6133 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw0-7b6759dd5d/isvc-sklearn-scale-raw-9167b-predictor-6c577cd946-458qh"
Oct 30 22:16:20.417652 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:16:20.417628    6133 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7b6759dd5d/isvc-sklearn-scale-raw-9167b-predictor-6c577cd946-458qh" podUID="acaaf46b-acde-4a9d-b95f-9081342e5009" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.29:8080: connect: connection refused"
Oct 30 22:16:21.420099 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:16:21.420064    6133 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7b6759dd5d/isvc-sklearn-scale-raw-9167b-predictor-6c577cd946-458qh" podUID="acaaf46b-acde-4a9d-b95f-9081342e5009" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.29:8080: connect: connection refused"
Oct 30 22:16:26.424731 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:16:26.424702    6133 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw0-7b6759dd5d/isvc-sklearn-scale-raw-9167b-predictor-6c577cd946-458qh"
Oct 30 22:16:26.425199 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:16:26.425174    6133 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7b6759dd5d/isvc-sklearn-scale-raw-9167b-predictor-6c577cd946-458qh" podUID="acaaf46b-acde-4a9d-b95f-9081342e5009" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.29:8080: connect: connection refused"
Oct 30 22:16:36.426131 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:16:36.426079    6133 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7b6759dd5d/isvc-sklearn-scale-raw-9167b-predictor-6c577cd946-458qh" podUID="acaaf46b-acde-4a9d-b95f-9081342e5009" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.29:8080: connect: connection refused"
Oct 30 22:16:39.527666 ip-10-0-143-246 kubenswrapper[6133]: E1030 22:16:39.527630    6133 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6jppj" containerName="extractor"
Oct 30 22:16:46.425170 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:16:46.425129    6133 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7b6759dd5d/isvc-sklearn-scale-raw-9167b-predictor-6c577cd946-458qh" podUID="acaaf46b-acde-4a9d-b95f-9081342e5009" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.29:8080: connect: connection refused"
Oct 30 22:16:56.425609 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:16:56.425562    6133 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7b6759dd5d/isvc-sklearn-scale-raw-9167b-predictor-6c577cd946-458qh" podUID="acaaf46b-acde-4a9d-b95f-9081342e5009" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.29:8080: connect: connection refused"
Oct 30 22:17:06.425348 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:17:06.425308    6133 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7b6759dd5d/isvc-sklearn-scale-raw-9167b-predictor-6c577cd946-458qh" podUID="acaaf46b-acde-4a9d-b95f-9081342e5009" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.29:8080: connect: connection refused"
Oct 30 22:17:09.527188 ip-10-0-143-246 kubenswrapper[6133]: E1030 22:17:09.527146    6133 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6jppj" containerName="extractor"
Oct 30 22:17:16.426021 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:17:16.425970    6133 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7b6759dd5d/isvc-sklearn-scale-raw-9167b-predictor-6c577cd946-458qh" podUID="acaaf46b-acde-4a9d-b95f-9081342e5009" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.29:8080: connect: connection refused"
Oct 30 22:17:26.425799 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:17:26.425759    6133 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7b6759dd5d/isvc-sklearn-scale-raw-9167b-predictor-6c577cd946-458qh" podUID="acaaf46b-acde-4a9d-b95f-9081342e5009" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.29:8080: connect: connection refused"
Oct 30 22:17:35.350874 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:17:35.350781    6133 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7b6759dd5d/isvc-sklearn-scale-raw-9167b-predictor-6c577cd946-458qh" podUID="acaaf46b-acde-4a9d-b95f-9081342e5009" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.29:8080: connect: connection refused"
Oct 30 22:17:39.527502 ip-10-0-143-246 kubenswrapper[6133]: E1030 22:17:39.527465    6133 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6jppj" containerName="extractor"
Oct 30 22:17:45.350664 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:17:45.350619    6133 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7b6759dd5d/isvc-sklearn-scale-raw-9167b-predictor-6c577cd946-458qh" podUID="acaaf46b-acde-4a9d-b95f-9081342e5009" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.29:8080: connect: connection refused"
Oct 30 22:17:54.946376 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:17:54.946337    6133 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-gw1-2e5a75ca98/isvc-canary-create-predictor-57d466f945-cc577"]
Oct 30 22:17:54.950024 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:17:54.950000    6133 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-2e5a75ca98/isvc-canary-create-predictor-57d466f945-cc577"
Oct 30 22:17:54.953790 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:17:54.953763    6133 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/5056aa18-e249-4a78-be47-cc8df47f7920-kserve-provision-location\") pod \"isvc-canary-create-predictor-57d466f945-cc577\" (UID: \"5056aa18-e249-4a78-be47-cc8df47f7920\") " pod="e2e-gw1-2e5a75ca98/isvc-canary-create-predictor-57d466f945-cc577"
Oct 30 22:17:54.953954 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:17:54.953800    6133 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/5056aa18-e249-4a78-be47-cc8df47f7920-cabundle-cert\") pod \"isvc-canary-create-predictor-57d466f945-cc577\" (UID: \"5056aa18-e249-4a78-be47-cc8df47f7920\") " pod="e2e-gw1-2e5a75ca98/isvc-canary-create-predictor-57d466f945-cc577"
Oct 30 22:17:54.954052 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:17:54.953955    6133 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fzqh7\" (UniqueName: \"kubernetes.io/projected/5056aa18-e249-4a78-be47-cc8df47f7920-kube-api-access-fzqh7\") pod \"isvc-canary-create-predictor-57d466f945-cc577\" (UID: \"5056aa18-e249-4a78-be47-cc8df47f7920\") " pod="e2e-gw1-2e5a75ca98/isvc-canary-create-predictor-57d466f945-cc577"
Oct 30 22:17:54.954117 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:17:54.954045    6133 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/5056aa18-e249-4a78-be47-cc8df47f7920-proxy-tls\") pod \"isvc-canary-create-predictor-57d466f945-cc577\" (UID: \"5056aa18-e249-4a78-be47-cc8df47f7920\") " pod="e2e-gw1-2e5a75ca98/isvc-canary-create-predictor-57d466f945-cc577"
Oct 30 22:17:54.954117 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:17:54.954079    6133 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/5056aa18-e249-4a78-be47-cc8df47f7920-kube-rbac-proxy-sar-config\") pod \"isvc-canary-create-predictor-57d466f945-cc577\" (UID: \"5056aa18-e249-4a78-be47-cc8df47f7920\") " pod="e2e-gw1-2e5a75ca98/isvc-canary-create-predictor-57d466f945-cc577"
Oct 30 22:17:54.982577 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:17:54.975775    6133 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw1-2e5a75ca98/isvc-canary-create-predictor-57d466f945-cc577"]
Oct 30 22:17:55.023735 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:17:55.023694    6133 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw1-2e5a75ca98\"/\"seaweedfs-s3-creds\""
Oct 30 22:17:55.023735 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:17:55.023752    6133 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw1-2e5a75ca98\"/\"isvc-canary-create-kube-rbac-proxy-sar-config\""
Oct 30 22:17:55.024159 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:17:55.023802    6133 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw1-2e5a75ca98\"/\"default-dockercfg-d9bw2\""
Oct 30 22:17:55.024159 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:17:55.023871    6133 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw1-2e5a75ca98\"/\"kube-root-ca.crt\""
Oct 30 22:17:55.024159 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:17:55.023946    6133 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw1-2e5a75ca98\"/\"openshift-service-ca.crt\""
Oct 30 22:17:55.024159 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:17:55.024055    6133 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw1-2e5a75ca98\"/\"odh-kserve-custom-ca-bundle\""
Oct 30 22:17:55.024159 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:17:55.024161    6133 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw1-2e5a75ca98\"/\"isvc-canary-create-predictor-serving-cert\""
Oct 30 22:17:55.054766 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:17:55.054721    6133 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/5056aa18-e249-4a78-be47-cc8df47f7920-proxy-tls\") pod \"isvc-canary-create-predictor-57d466f945-cc577\" (UID: \"5056aa18-e249-4a78-be47-cc8df47f7920\") " pod="e2e-gw1-2e5a75ca98/isvc-canary-create-predictor-57d466f945-cc577"
Oct 30 22:17:55.054766 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:17:55.054765    6133 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/5056aa18-e249-4a78-be47-cc8df47f7920-kube-rbac-proxy-sar-config\") pod \"isvc-canary-create-predictor-57d466f945-cc577\" (UID: \"5056aa18-e249-4a78-be47-cc8df47f7920\") " pod="e2e-gw1-2e5a75ca98/isvc-canary-create-predictor-57d466f945-cc577"
Oct 30 22:17:55.055081 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:17:55.054805    6133 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/5056aa18-e249-4a78-be47-cc8df47f7920-kserve-provision-location\") pod \"isvc-canary-create-predictor-57d466f945-cc577\" (UID: \"5056aa18-e249-4a78-be47-cc8df47f7920\") " pod="e2e-gw1-2e5a75ca98/isvc-canary-create-predictor-57d466f945-cc577"
Oct 30 22:17:55.055081 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:17:55.054844    6133 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/5056aa18-e249-4a78-be47-cc8df47f7920-cabundle-cert\") pod \"isvc-canary-create-predictor-57d466f945-cc577\" (UID: \"5056aa18-e249-4a78-be47-cc8df47f7920\") " pod="e2e-gw1-2e5a75ca98/isvc-canary-create-predictor-57d466f945-cc577"
Oct 30 22:17:55.055081 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:17:55.054906    6133 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fzqh7\" (UniqueName: \"kubernetes.io/projected/5056aa18-e249-4a78-be47-cc8df47f7920-kube-api-access-fzqh7\") pod \"isvc-canary-create-predictor-57d466f945-cc577\" (UID: \"5056aa18-e249-4a78-be47-cc8df47f7920\") " pod="e2e-gw1-2e5a75ca98/isvc-canary-create-predictor-57d466f945-cc577"
Oct 30 22:17:55.055399 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:17:55.055370    6133 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/5056aa18-e249-4a78-be47-cc8df47f7920-kserve-provision-location\") pod \"isvc-canary-create-predictor-57d466f945-cc577\" (UID: \"5056aa18-e249-4a78-be47-cc8df47f7920\") " pod="e2e-gw1-2e5a75ca98/isvc-canary-create-predictor-57d466f945-cc577"
Oct 30 22:17:55.055725 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:17:55.055702    6133 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/5056aa18-e249-4a78-be47-cc8df47f7920-kube-rbac-proxy-sar-config\") pod \"isvc-canary-create-predictor-57d466f945-cc577\" (UID: \"5056aa18-e249-4a78-be47-cc8df47f7920\") " pod="e2e-gw1-2e5a75ca98/isvc-canary-create-predictor-57d466f945-cc577"
Oct 30 22:17:55.055809 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:17:55.055710    6133 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/5056aa18-e249-4a78-be47-cc8df47f7920-cabundle-cert\") pod \"isvc-canary-create-predictor-57d466f945-cc577\" (UID: \"5056aa18-e249-4a78-be47-cc8df47f7920\") " pod="e2e-gw1-2e5a75ca98/isvc-canary-create-predictor-57d466f945-cc577"
Oct 30 22:17:55.057271 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:17:55.057255    6133 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/5056aa18-e249-4a78-be47-cc8df47f7920-proxy-tls\") pod \"isvc-canary-create-predictor-57d466f945-cc577\" (UID: \"5056aa18-e249-4a78-be47-cc8df47f7920\") " pod="e2e-gw1-2e5a75ca98/isvc-canary-create-predictor-57d466f945-cc577"
Oct 30 22:17:55.077228 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:17:55.077198    6133 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fzqh7\" (UniqueName: \"kubernetes.io/projected/5056aa18-e249-4a78-be47-cc8df47f7920-kube-api-access-fzqh7\") pod \"isvc-canary-create-predictor-57d466f945-cc577\" (UID: \"5056aa18-e249-4a78-be47-cc8df47f7920\") " pod="e2e-gw1-2e5a75ca98/isvc-canary-create-predictor-57d466f945-cc577"
Oct 30 22:17:55.268041 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:17:55.267978    6133 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-2e5a75ca98/isvc-canary-create-predictor-57d466f945-cc577"
Oct 30 22:17:55.350780 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:17:55.350737    6133 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7b6759dd5d/isvc-sklearn-scale-raw-9167b-predictor-6c577cd946-458qh" podUID="acaaf46b-acde-4a9d-b95f-9081342e5009" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.29:8080: connect: connection refused"
Oct 30 22:17:55.576099 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:17:55.576016    6133 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw1-2e5a75ca98/isvc-canary-create-predictor-57d466f945-cc577"]
Oct 30 22:17:55.578914 ip-10-0-143-246 kubenswrapper[6133]: W1030 22:17:55.578885    6133 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5056aa18_e249_4a78_be47_cc8df47f7920.slice/crio-4d459c89e670f48d63d9a13c640475470a27127acbaa95d87594b1852a54b979 WatchSource:0}: Error finding container 4d459c89e670f48d63d9a13c640475470a27127acbaa95d87594b1852a54b979: Status 404 returned error can't find the container with id 4d459c89e670f48d63d9a13c640475470a27127acbaa95d87594b1852a54b979
Oct 30 22:17:55.753027 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:17:55.752965    6133 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-2e5a75ca98/isvc-canary-create-predictor-57d466f945-cc577" event={"ID":"5056aa18-e249-4a78-be47-cc8df47f7920","Type":"ContainerStarted","Data":"53028db89edb071241883cd39fba0e1db7e0d9d16dbbc9f1a1712be0dcd23e70"}
Oct 30 22:17:55.753202 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:17:55.753061    6133 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-2e5a75ca98/isvc-canary-create-predictor-57d466f945-cc577" event={"ID":"5056aa18-e249-4a78-be47-cc8df47f7920","Type":"ContainerStarted","Data":"4d459c89e670f48d63d9a13c640475470a27127acbaa95d87594b1852a54b979"}
Oct 30 22:17:59.768983 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:17:59.768948    6133 generic.go:356] "Generic (PLEG): container finished" podID="5056aa18-e249-4a78-be47-cc8df47f7920" containerID="53028db89edb071241883cd39fba0e1db7e0d9d16dbbc9f1a1712be0dcd23e70" exitCode=0
Oct 30 22:17:59.769427 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:17:59.769019    6133 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-2e5a75ca98/isvc-canary-create-predictor-57d466f945-cc577" event={"ID":"5056aa18-e249-4a78-be47-cc8df47f7920","Type":"ContainerDied","Data":"53028db89edb071241883cd39fba0e1db7e0d9d16dbbc9f1a1712be0dcd23e70"}
Oct 30 22:18:00.775950 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:18:00.775917    6133 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-2e5a75ca98/isvc-canary-create-predictor-57d466f945-cc577" event={"ID":"5056aa18-e249-4a78-be47-cc8df47f7920","Type":"ContainerStarted","Data":"e3cd9afbeab656a32aabfa4260a549dcf8a0d3d299d6eefe8a0b56c4ee148b11"}
Oct 30 22:18:00.776354 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:18:00.775978    6133 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-2e5a75ca98/isvc-canary-create-predictor-57d466f945-cc577" event={"ID":"5056aa18-e249-4a78-be47-cc8df47f7920","Type":"ContainerStarted","Data":"59f545d1f37628359be89fe98525303f1090c1ff19041c990d0308102edfd5b3"}
Oct 30 22:18:00.776354 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:18:00.776174    6133 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw1-2e5a75ca98/isvc-canary-create-predictor-57d466f945-cc577"
Oct 30 22:18:00.832374 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:18:00.832317    6133 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-gw1-2e5a75ca98/isvc-canary-create-predictor-57d466f945-cc577" podStartSLOduration=2.977196865 podStartE2EDuration="6.832303994s" totalImagesPullingTime="0s" totalInitContainerRuntime="3.855107129s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:17:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:18:00.831494802 +0000 UTC m=+1387.592250716" watchObservedRunningTime="2026-10-30 22:18:00.832303994 +0000 UTC m=+1387.593059874"
Oct 30 22:18:01.779621 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:18:01.779583    6133 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw1-2e5a75ca98/isvc-canary-create-predictor-57d466f945-cc577"
Oct 30 22:18:01.781028 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:18:01.780999    6133 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-2e5a75ca98/isvc-canary-create-predictor-57d466f945-cc577" podUID="5056aa18-e249-4a78-be47-cc8df47f7920" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.30:8080: connect: connection refused"
Oct 30 22:18:02.783658 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:18:02.783619    6133 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-2e5a75ca98/isvc-canary-create-predictor-57d466f945-cc577" podUID="5056aa18-e249-4a78-be47-cc8df47f7920" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.30:8080: connect: connection refused"
Oct 30 22:18:05.350896 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:18:05.350848    6133 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7b6759dd5d/isvc-sklearn-scale-raw-9167b-predictor-6c577cd946-458qh" podUID="acaaf46b-acde-4a9d-b95f-9081342e5009" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.29:8080: connect: connection refused"
Oct 30 22:18:07.788444 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:18:07.788415    6133 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw1-2e5a75ca98/isvc-canary-create-predictor-57d466f945-cc577"
Oct 30 22:18:07.789059 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:18:07.789034    6133 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-2e5a75ca98/isvc-canary-create-predictor-57d466f945-cc577" podUID="5056aa18-e249-4a78-be47-cc8df47f7920" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.30:8080: connect: connection refused"
Oct 30 22:18:09.527304 ip-10-0-143-246 kubenswrapper[6133]: E1030 22:18:09.527266    6133 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6jppj" containerName="extractor"
Oct 30 22:18:15.350718 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:18:15.350668    6133 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7b6759dd5d/isvc-sklearn-scale-raw-9167b-predictor-6c577cd946-458qh" podUID="acaaf46b-acde-4a9d-b95f-9081342e5009" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.29:8080: connect: connection refused"
Oct 30 22:18:17.789000 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:18:17.788948    6133 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-2e5a75ca98/isvc-canary-create-predictor-57d466f945-cc577" podUID="5056aa18-e249-4a78-be47-cc8df47f7920" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.30:8080: connect: connection refused"
Oct 30 22:18:25.350929 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:18:25.350879    6133 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7b6759dd5d/isvc-sklearn-scale-raw-9167b-predictor-6c577cd946-458qh" podUID="acaaf46b-acde-4a9d-b95f-9081342e5009" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.29:8080: connect: connection refused"
Oct 30 22:18:27.790037 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:18:27.789970    6133 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-2e5a75ca98/isvc-canary-create-predictor-57d466f945-cc577" podUID="5056aa18-e249-4a78-be47-cc8df47f7920" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.30:8080: connect: connection refused"
Oct 30 22:18:35.350717 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:18:35.350667    6133 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7b6759dd5d/isvc-sklearn-scale-raw-9167b-predictor-6c577cd946-458qh" podUID="acaaf46b-acde-4a9d-b95f-9081342e5009" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.29:8080: connect: connection refused"
Oct 30 22:18:37.789747 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:18:37.789701    6133 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-2e5a75ca98/isvc-canary-create-predictor-57d466f945-cc577" podUID="5056aa18-e249-4a78-be47-cc8df47f7920" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.30:8080: connect: connection refused"
Oct 30 22:18:39.528898 ip-10-0-143-246 kubenswrapper[6133]: E1030 22:18:39.528854    6133 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6jppj" containerName="extractor"
Oct 30 22:18:45.350821 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:18:45.350772    6133 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7b6759dd5d/isvc-sklearn-scale-raw-9167b-predictor-6c577cd946-458qh" podUID="acaaf46b-acde-4a9d-b95f-9081342e5009" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.29:8080: connect: connection refused"
Oct 30 22:18:47.789891 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:18:47.789850    6133 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-2e5a75ca98/isvc-canary-create-predictor-57d466f945-cc577" podUID="5056aa18-e249-4a78-be47-cc8df47f7920" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.30:8080: connect: connection refused"
Oct 30 22:18:55.351663 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:18:55.351612    6133 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7b6759dd5d/isvc-sklearn-scale-raw-9167b-predictor-6c577cd946-458qh" podUID="acaaf46b-acde-4a9d-b95f-9081342e5009" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.29:8080: connect: connection refused"
Oct 30 22:18:56.351593 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:18:56.351548    6133 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7b6759dd5d/isvc-sklearn-scale-raw-9167b-predictor-6c577cd946-458qh" podUID="acaaf46b-acde-4a9d-b95f-9081342e5009" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.29:8080: connect: connection refused"
Oct 30 22:18:57.789654 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:18:57.789616    6133 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-2e5a75ca98/isvc-canary-create-predictor-57d466f945-cc577" podUID="5056aa18-e249-4a78-be47-cc8df47f7920" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.30:8080: connect: connection refused"
Oct 30 22:19:06.353173 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:19:06.353089    6133 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw0-7b6759dd5d/isvc-sklearn-scale-raw-9167b-predictor-6c577cd946-458qh"
Oct 30 22:19:07.789610 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:19:07.789569    6133 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-2e5a75ca98/isvc-canary-create-predictor-57d466f945-cc577" podUID="5056aa18-e249-4a78-be47-cc8df47f7920" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.30:8080: connect: connection refused"
Oct 30 22:19:09.526818 ip-10-0-143-246 kubenswrapper[6133]: E1030 22:19:09.526783    6133 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6jppj" containerName="extractor"
Oct 30 22:19:14.501951 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:19:14.501918    6133 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw0-7b6759dd5d/isvc-sklearn-scale-raw-9167b-predictor-6c577cd946-458qh"]
Oct 30 22:19:14.502386 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:19:14.502197    6133 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw0-7b6759dd5d/isvc-sklearn-scale-raw-9167b-predictor-6c577cd946-458qh" podUID="acaaf46b-acde-4a9d-b95f-9081342e5009" containerName="kserve-container" containerID="cri-o://dc09e1be0cf893de26c652d4708a4c43e5ce6a250d68700bf29068d258984d26" gracePeriod=30
Oct 30 22:19:14.502386 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:19:14.502208    6133 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw0-7b6759dd5d/isvc-sklearn-scale-raw-9167b-predictor-6c577cd946-458qh" podUID="acaaf46b-acde-4a9d-b95f-9081342e5009" containerName="kube-rbac-proxy" containerID="cri-o://3150a3723af688a2f690829e7dde75140e5b41f1d10adfda24358c3156cb2746" gracePeriod=30
Oct 30 22:19:15.048559 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:19:15.048521    6133 generic.go:356] "Generic (PLEG): container finished" podID="acaaf46b-acde-4a9d-b95f-9081342e5009" containerID="3150a3723af688a2f690829e7dde75140e5b41f1d10adfda24358c3156cb2746" exitCode=0
Oct 30 22:19:15.048744 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:19:15.048589    6133 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-7b6759dd5d/isvc-sklearn-scale-raw-9167b-predictor-6c577cd946-458qh" event={"ID":"acaaf46b-acde-4a9d-b95f-9081342e5009","Type":"ContainerDied","Data":"3150a3723af688a2f690829e7dde75140e5b41f1d10adfda24358c3156cb2746"}
Oct 30 22:19:16.352143 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:19:16.352104    6133 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7b6759dd5d/isvc-sklearn-scale-raw-9167b-predictor-6c577cd946-458qh" podUID="acaaf46b-acde-4a9d-b95f-9081342e5009" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.29:8080: connect: connection refused"
Oct 30 22:19:16.420931 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:19:16.420885    6133 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7b6759dd5d/isvc-sklearn-scale-raw-9167b-predictor-6c577cd946-458qh" podUID="acaaf46b-acde-4a9d-b95f-9081342e5009" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.29:8643/healthz\": dial tcp 10.134.0.29:8643: connect: connection refused"
Oct 30 22:19:17.789486 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:19:17.789447    6133 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-2e5a75ca98/isvc-canary-create-predictor-57d466f945-cc577" podUID="5056aa18-e249-4a78-be47-cc8df47f7920" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.30:8080: connect: connection refused"
Oct 30 22:19:21.420716 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:19:21.420668    6133 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-7b6759dd5d/isvc-sklearn-scale-raw-9167b-predictor-6c577cd946-458qh" podUID="acaaf46b-acde-4a9d-b95f-9081342e5009" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.29:8643/healthz\": dial tcp 10.134.0.29:8643: connect: connection refused"
Oct 30 22:19:22.351860 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:19:22.351814    6133 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw1-2e5a75ca98/isvc-canary-create-predictor-57d466f945-cc577"
Oct 30 22:19:25.022799 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:19:25.022762    6133 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw1-2e5a75ca98/isvc-canary-create-predictor-57d466f945-cc577"]
Oct 30 22:19:25.023327 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:19:25.022980    6133 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw1-2e5a75ca98/isvc-canary-create-predictor-57d466f945-cc577" podUID="5056aa18-e249-4a78-be47-cc8df47f7920" containerName="kserve-container" containerID="cri-o://59f545d1f37628359be89fe98525303f1090c1ff19041c990d0308102edfd5b3" gracePeriod=30
Oct 30 22:19:25.023327 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:19:25.023051    6133 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw1-2e5a75ca98/isvc-canary-create-predictor-57d466f945-cc577" podUID="5056aa18-e249-4a78-be47-cc8df47f7920" containerName="kube-rbac-proxy" containerID="cri-o://e3cd9afbeab656a32aabfa4260a549dcf8a0d3d299d6eefe8a0b56c4ee148b11" gracePeriod=30
Oct 30 22:19:25.254592 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:19:25.254564    6133 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-7b6759dd5d/isvc-sklearn-scale-raw-9167b-predictor-6c577cd946-458qh"
Oct 30 22:19:25.401094 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:19:25.400984    6133 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/acaaf46b-acde-4a9d-b95f-9081342e5009-kube-api-access-lq79t\" (UniqueName: \"kubernetes.io/projected/acaaf46b-acde-4a9d-b95f-9081342e5009-kube-api-access-lq79t\") pod \"acaaf46b-acde-4a9d-b95f-9081342e5009\" (UID: \"acaaf46b-acde-4a9d-b95f-9081342e5009\") "
Oct 30 22:19:25.401277 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:19:25.401114    6133 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/acaaf46b-acde-4a9d-b95f-9081342e5009-cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/acaaf46b-acde-4a9d-b95f-9081342e5009-cabundle-cert\") pod \"acaaf46b-acde-4a9d-b95f-9081342e5009\" (UID: \"acaaf46b-acde-4a9d-b95f-9081342e5009\") "
Oct 30 22:19:25.401277 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:19:25.401149    6133 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/acaaf46b-acde-4a9d-b95f-9081342e5009-proxy-tls\" (UniqueName: \"kubernetes.io/secret/acaaf46b-acde-4a9d-b95f-9081342e5009-proxy-tls\") pod \"acaaf46b-acde-4a9d-b95f-9081342e5009\" (UID: \"acaaf46b-acde-4a9d-b95f-9081342e5009\") "
Oct 30 22:19:25.401277 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:19:25.401169    6133 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/acaaf46b-acde-4a9d-b95f-9081342e5009-kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/acaaf46b-acde-4a9d-b95f-9081342e5009-kserve-provision-location\") pod \"acaaf46b-acde-4a9d-b95f-9081342e5009\" (UID: \"acaaf46b-acde-4a9d-b95f-9081342e5009\") "
Oct 30 22:19:25.401277 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:19:25.401206    6133 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/acaaf46b-acde-4a9d-b95f-9081342e5009-kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/acaaf46b-acde-4a9d-b95f-9081342e5009-kube-rbac-proxy-sar-config\") pod \"acaaf46b-acde-4a9d-b95f-9081342e5009\" (UID: \"acaaf46b-acde-4a9d-b95f-9081342e5009\") "
Oct 30 22:19:25.401531 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:19:25.401498    6133 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/acaaf46b-acde-4a9d-b95f-9081342e5009-cabundle-cert" pod "acaaf46b-acde-4a9d-b95f-9081342e5009" (UID: "acaaf46b-acde-4a9d-b95f-9081342e5009"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:19:25.401531 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:19:25.401522    6133 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/acaaf46b-acde-4a9d-b95f-9081342e5009-kserve-provision-location" pod "acaaf46b-acde-4a9d-b95f-9081342e5009" (UID: "acaaf46b-acde-4a9d-b95f-9081342e5009"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:19:25.401693 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:19:25.401638    6133 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/acaaf46b-acde-4a9d-b95f-9081342e5009-kube-rbac-proxy-sar-config" pod "acaaf46b-acde-4a9d-b95f-9081342e5009" (UID: "acaaf46b-acde-4a9d-b95f-9081342e5009"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:19:25.403304 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:19:25.403278    6133 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/acaaf46b-acde-4a9d-b95f-9081342e5009-kube-api-access-lq79t" pod "acaaf46b-acde-4a9d-b95f-9081342e5009" (UID: "acaaf46b-acde-4a9d-b95f-9081342e5009"). InnerVolumeSpecName "kube-api-access-lq79t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:19:25.403304 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:19:25.403285    6133 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/acaaf46b-acde-4a9d-b95f-9081342e5009-proxy-tls" pod "acaaf46b-acde-4a9d-b95f-9081342e5009" (UID: "acaaf46b-acde-4a9d-b95f-9081342e5009"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:19:25.502114 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:19:25.502072    6133 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/acaaf46b-acde-4a9d-b95f-9081342e5009-cabundle-cert\") on node \"ip-10-0-143-246.ec2.internal\" DevicePath \"\""
Oct 30 22:19:25.502114 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:19:25.502107    6133 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/acaaf46b-acde-4a9d-b95f-9081342e5009-proxy-tls\") on node \"ip-10-0-143-246.ec2.internal\" DevicePath \"\""
Oct 30 22:19:25.502114 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:19:25.502119    6133 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/acaaf46b-acde-4a9d-b95f-9081342e5009-kserve-provision-location\") on node \"ip-10-0-143-246.ec2.internal\" DevicePath \"\""
Oct 30 22:19:25.502358 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:19:25.502133    6133 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/acaaf46b-acde-4a9d-b95f-9081342e5009-kube-rbac-proxy-sar-config\") on node \"ip-10-0-143-246.ec2.internal\" DevicePath \"\""
Oct 30 22:19:25.502358 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:19:25.502145    6133 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lq79t\" (UniqueName: \"kubernetes.io/projected/acaaf46b-acde-4a9d-b95f-9081342e5009-kube-api-access-lq79t\") on node \"ip-10-0-143-246.ec2.internal\" DevicePath \"\""
Oct 30 22:19:26.090918 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:19:26.090882    6133 generic.go:356] "Generic (PLEG): container finished" podID="acaaf46b-acde-4a9d-b95f-9081342e5009" containerID="dc09e1be0cf893de26c652d4708a4c43e5ce6a250d68700bf29068d258984d26" exitCode=0
Oct 30 22:19:26.091442 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:19:26.090950    6133 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-7b6759dd5d/isvc-sklearn-scale-raw-9167b-predictor-6c577cd946-458qh"
Oct 30 22:19:26.091442 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:19:26.090971    6133 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-7b6759dd5d/isvc-sklearn-scale-raw-9167b-predictor-6c577cd946-458qh" event={"ID":"acaaf46b-acde-4a9d-b95f-9081342e5009","Type":"ContainerDied","Data":"dc09e1be0cf893de26c652d4708a4c43e5ce6a250d68700bf29068d258984d26"}
Oct 30 22:19:26.091442 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:19:26.091053    6133 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-7b6759dd5d/isvc-sklearn-scale-raw-9167b-predictor-6c577cd946-458qh" event={"ID":"acaaf46b-acde-4a9d-b95f-9081342e5009","Type":"ContainerDied","Data":"14229531744c890faf537eab1fcb26c5ae5a23cad7e2fd30db99dc6cb8b2bec5"}
Oct 30 22:19:26.091442 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:19:26.091086    6133 scope.go:122] "RemoveContainer" containerID="3150a3723af688a2f690829e7dde75140e5b41f1d10adfda24358c3156cb2746"
Oct 30 22:19:26.093413 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:19:26.093379    6133 generic.go:356] "Generic (PLEG): container finished" podID="5056aa18-e249-4a78-be47-cc8df47f7920" containerID="e3cd9afbeab656a32aabfa4260a549dcf8a0d3d299d6eefe8a0b56c4ee148b11" exitCode=0
Oct 30 22:19:26.093542 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:19:26.093408    6133 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-2e5a75ca98/isvc-canary-create-predictor-57d466f945-cc577" event={"ID":"5056aa18-e249-4a78-be47-cc8df47f7920","Type":"ContainerDied","Data":"e3cd9afbeab656a32aabfa4260a549dcf8a0d3d299d6eefe8a0b56c4ee148b11"}
Oct 30 22:19:26.099811 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:19:26.099792    6133 scope.go:122] "RemoveContainer" containerID="dc09e1be0cf893de26c652d4708a4c43e5ce6a250d68700bf29068d258984d26"
Oct 30 22:19:26.107924 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:19:26.107905    6133 scope.go:122] "RemoveContainer" containerID="6e79f0dfed2e68c95d842f654ea69ba5eb378f6afe9ce7a4540670674c487600"
Oct 30 22:19:26.115437 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:19:26.115416    6133 scope.go:122] "RemoveContainer" containerID="3150a3723af688a2f690829e7dde75140e5b41f1d10adfda24358c3156cb2746"
Oct 30 22:19:26.115759 ip-10-0-143-246 kubenswrapper[6133]: E1030 22:19:26.115739    6133 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3150a3723af688a2f690829e7dde75140e5b41f1d10adfda24358c3156cb2746\": container with ID starting with 3150a3723af688a2f690829e7dde75140e5b41f1d10adfda24358c3156cb2746 not found: ID does not exist" containerID="3150a3723af688a2f690829e7dde75140e5b41f1d10adfda24358c3156cb2746"
Oct 30 22:19:26.115804 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:19:26.115770    6133 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3150a3723af688a2f690829e7dde75140e5b41f1d10adfda24358c3156cb2746"} err="failed to get container status \"3150a3723af688a2f690829e7dde75140e5b41f1d10adfda24358c3156cb2746\": rpc error: code = NotFound desc = could not find container \"3150a3723af688a2f690829e7dde75140e5b41f1d10adfda24358c3156cb2746\": container with ID starting with 3150a3723af688a2f690829e7dde75140e5b41f1d10adfda24358c3156cb2746 not found: ID does not exist"
Oct 30 22:19:26.115804 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:19:26.115789    6133 scope.go:122] "RemoveContainer" containerID="dc09e1be0cf893de26c652d4708a4c43e5ce6a250d68700bf29068d258984d26"
Oct 30 22:19:26.116056 ip-10-0-143-246 kubenswrapper[6133]: E1030 22:19:26.116038    6133 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dc09e1be0cf893de26c652d4708a4c43e5ce6a250d68700bf29068d258984d26\": container with ID starting with dc09e1be0cf893de26c652d4708a4c43e5ce6a250d68700bf29068d258984d26 not found: ID does not exist" containerID="dc09e1be0cf893de26c652d4708a4c43e5ce6a250d68700bf29068d258984d26"
Oct 30 22:19:26.116093 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:19:26.116065    6133 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dc09e1be0cf893de26c652d4708a4c43e5ce6a250d68700bf29068d258984d26"} err="failed to get container status \"dc09e1be0cf893de26c652d4708a4c43e5ce6a250d68700bf29068d258984d26\": rpc error: code = NotFound desc = could not find container \"dc09e1be0cf893de26c652d4708a4c43e5ce6a250d68700bf29068d258984d26\": container with ID starting with dc09e1be0cf893de26c652d4708a4c43e5ce6a250d68700bf29068d258984d26 not found: ID does not exist"
Oct 30 22:19:26.116093 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:19:26.116080    6133 scope.go:122] "RemoveContainer" containerID="6e79f0dfed2e68c95d842f654ea69ba5eb378f6afe9ce7a4540670674c487600"
Oct 30 22:19:26.116295 ip-10-0-143-246 kubenswrapper[6133]: E1030 22:19:26.116279    6133 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6e79f0dfed2e68c95d842f654ea69ba5eb378f6afe9ce7a4540670674c487600\": container with ID starting with 6e79f0dfed2e68c95d842f654ea69ba5eb378f6afe9ce7a4540670674c487600 not found: ID does not exist" containerID="6e79f0dfed2e68c95d842f654ea69ba5eb378f6afe9ce7a4540670674c487600"
Oct 30 22:19:26.116333 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:19:26.116301    6133 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6e79f0dfed2e68c95d842f654ea69ba5eb378f6afe9ce7a4540670674c487600"} err="failed to get container status \"6e79f0dfed2e68c95d842f654ea69ba5eb378f6afe9ce7a4540670674c487600\": rpc error: code = NotFound desc = could not find container \"6e79f0dfed2e68c95d842f654ea69ba5eb378f6afe9ce7a4540670674c487600\": container with ID starting with 6e79f0dfed2e68c95d842f654ea69ba5eb378f6afe9ce7a4540670674c487600 not found: ID does not exist"
Oct 30 22:19:26.158269 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:19:26.158240    6133 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw0-7b6759dd5d/isvc-sklearn-scale-raw-9167b-predictor-6c577cd946-458qh"]
Oct 30 22:19:26.166174 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:19:26.166150    6133 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-gw0-7b6759dd5d/isvc-sklearn-scale-raw-9167b-predictor-6c577cd946-458qh"]
Oct 30 22:19:27.354642 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:19:27.354610    6133 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="acaaf46b-acde-4a9d-b95f-9081342e5009" path="/var/lib/kubelet/pods/acaaf46b-acde-4a9d-b95f-9081342e5009/volumes"
Oct 30 22:19:27.784407 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:19:27.784366    6133 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-2e5a75ca98/isvc-canary-create-predictor-57d466f945-cc577" podUID="5056aa18-e249-4a78-be47-cc8df47f7920" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.30:8643/healthz\": dial tcp 10.134.0.30:8643: connect: connection refused"
Oct 30 22:19:30.362795 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:19:30.362770    6133 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-2e5a75ca98/isvc-canary-create-predictor-57d466f945-cc577"
Oct 30 22:19:30.444954 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:19:30.444876    6133 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/5056aa18-e249-4a78-be47-cc8df47f7920-kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/5056aa18-e249-4a78-be47-cc8df47f7920-kube-rbac-proxy-sar-config\") pod \"5056aa18-e249-4a78-be47-cc8df47f7920\" (UID: \"5056aa18-e249-4a78-be47-cc8df47f7920\") "
Oct 30 22:19:30.445139 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:19:30.444969    6133 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5056aa18-e249-4a78-be47-cc8df47f7920-kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/5056aa18-e249-4a78-be47-cc8df47f7920-kserve-provision-location\") pod \"5056aa18-e249-4a78-be47-cc8df47f7920\" (UID: \"5056aa18-e249-4a78-be47-cc8df47f7920\") "
Oct 30 22:19:30.445139 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:19:30.445025    6133 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5056aa18-e249-4a78-be47-cc8df47f7920-kube-api-access-fzqh7\" (UniqueName: \"kubernetes.io/projected/5056aa18-e249-4a78-be47-cc8df47f7920-kube-api-access-fzqh7\") pod \"5056aa18-e249-4a78-be47-cc8df47f7920\" (UID: \"5056aa18-e249-4a78-be47-cc8df47f7920\") "
Oct 30 22:19:30.445139 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:19:30.445094    6133 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/5056aa18-e249-4a78-be47-cc8df47f7920-cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/5056aa18-e249-4a78-be47-cc8df47f7920-cabundle-cert\") pod \"5056aa18-e249-4a78-be47-cc8df47f7920\" (UID: \"5056aa18-e249-4a78-be47-cc8df47f7920\") "
Oct 30 22:19:30.445326 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:19:30.445140    6133 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/5056aa18-e249-4a78-be47-cc8df47f7920-proxy-tls\" (UniqueName: \"kubernetes.io/secret/5056aa18-e249-4a78-be47-cc8df47f7920-proxy-tls\") pod \"5056aa18-e249-4a78-be47-cc8df47f7920\" (UID: \"5056aa18-e249-4a78-be47-cc8df47f7920\") "
Oct 30 22:19:30.445326 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:19:30.445282    6133 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5056aa18-e249-4a78-be47-cc8df47f7920-kserve-provision-location" pod "5056aa18-e249-4a78-be47-cc8df47f7920" (UID: "5056aa18-e249-4a78-be47-cc8df47f7920"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:19:30.445326 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:19:30.445285    6133 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5056aa18-e249-4a78-be47-cc8df47f7920-kube-rbac-proxy-sar-config" pod "5056aa18-e249-4a78-be47-cc8df47f7920" (UID: "5056aa18-e249-4a78-be47-cc8df47f7920"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:19:30.445533 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:19:30.445490    6133 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/5056aa18-e249-4a78-be47-cc8df47f7920-kserve-provision-location\") on node \"ip-10-0-143-246.ec2.internal\" DevicePath \"\""
Oct 30 22:19:30.445533 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:19:30.445511    6133 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/5056aa18-e249-4a78-be47-cc8df47f7920-kube-rbac-proxy-sar-config\") on node \"ip-10-0-143-246.ec2.internal\" DevicePath \"\""
Oct 30 22:19:30.445646 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:19:30.445546    6133 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5056aa18-e249-4a78-be47-cc8df47f7920-cabundle-cert" pod "5056aa18-e249-4a78-be47-cc8df47f7920" (UID: "5056aa18-e249-4a78-be47-cc8df47f7920"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:19:30.447243 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:19:30.447218    6133 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5056aa18-e249-4a78-be47-cc8df47f7920-proxy-tls" pod "5056aa18-e249-4a78-be47-cc8df47f7920" (UID: "5056aa18-e249-4a78-be47-cc8df47f7920"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:19:30.447384 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:19:30.447320    6133 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5056aa18-e249-4a78-be47-cc8df47f7920-kube-api-access-fzqh7" pod "5056aa18-e249-4a78-be47-cc8df47f7920" (UID: "5056aa18-e249-4a78-be47-cc8df47f7920"). InnerVolumeSpecName "kube-api-access-fzqh7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:19:30.546042 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:19:30.545964    6133 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fzqh7\" (UniqueName: \"kubernetes.io/projected/5056aa18-e249-4a78-be47-cc8df47f7920-kube-api-access-fzqh7\") on node \"ip-10-0-143-246.ec2.internal\" DevicePath \"\""
Oct 30 22:19:30.546042 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:19:30.546031    6133 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/5056aa18-e249-4a78-be47-cc8df47f7920-cabundle-cert\") on node \"ip-10-0-143-246.ec2.internal\" DevicePath \"\""
Oct 30 22:19:30.546042 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:19:30.546041    6133 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/5056aa18-e249-4a78-be47-cc8df47f7920-proxy-tls\") on node \"ip-10-0-143-246.ec2.internal\" DevicePath \"\""
Oct 30 22:19:31.113375 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:19:31.113336    6133 generic.go:356] "Generic (PLEG): container finished" podID="5056aa18-e249-4a78-be47-cc8df47f7920" containerID="59f545d1f37628359be89fe98525303f1090c1ff19041c990d0308102edfd5b3" exitCode=0
Oct 30 22:19:31.113549 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:19:31.113382    6133 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-2e5a75ca98/isvc-canary-create-predictor-57d466f945-cc577" event={"ID":"5056aa18-e249-4a78-be47-cc8df47f7920","Type":"ContainerDied","Data":"59f545d1f37628359be89fe98525303f1090c1ff19041c990d0308102edfd5b3"}
Oct 30 22:19:31.113549 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:19:31.113408    6133 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-2e5a75ca98/isvc-canary-create-predictor-57d466f945-cc577"
Oct 30 22:19:31.113549 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:19:31.113452    6133 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-2e5a75ca98/isvc-canary-create-predictor-57d466f945-cc577" event={"ID":"5056aa18-e249-4a78-be47-cc8df47f7920","Type":"ContainerDied","Data":"4d459c89e670f48d63d9a13c640475470a27127acbaa95d87594b1852a54b979"}
Oct 30 22:19:31.113549 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:19:31.113478    6133 scope.go:122] "RemoveContainer" containerID="e3cd9afbeab656a32aabfa4260a549dcf8a0d3d299d6eefe8a0b56c4ee148b11"
Oct 30 22:19:31.123041 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:19:31.123018    6133 scope.go:122] "RemoveContainer" containerID="59f545d1f37628359be89fe98525303f1090c1ff19041c990d0308102edfd5b3"
Oct 30 22:19:31.130215 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:19:31.130193    6133 scope.go:122] "RemoveContainer" containerID="53028db89edb071241883cd39fba0e1db7e0d9d16dbbc9f1a1712be0dcd23e70"
Oct 30 22:19:31.137189 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:19:31.137171    6133 scope.go:122] "RemoveContainer" containerID="e3cd9afbeab656a32aabfa4260a549dcf8a0d3d299d6eefe8a0b56c4ee148b11"
Oct 30 22:19:31.137465 ip-10-0-143-246 kubenswrapper[6133]: E1030 22:19:31.137440    6133 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e3cd9afbeab656a32aabfa4260a549dcf8a0d3d299d6eefe8a0b56c4ee148b11\": container with ID starting with e3cd9afbeab656a32aabfa4260a549dcf8a0d3d299d6eefe8a0b56c4ee148b11 not found: ID does not exist" containerID="e3cd9afbeab656a32aabfa4260a549dcf8a0d3d299d6eefe8a0b56c4ee148b11"
Oct 30 22:19:31.137525 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:19:31.137474    6133 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e3cd9afbeab656a32aabfa4260a549dcf8a0d3d299d6eefe8a0b56c4ee148b11"} err="failed to get container status \"e3cd9afbeab656a32aabfa4260a549dcf8a0d3d299d6eefe8a0b56c4ee148b11\": rpc error: code = NotFound desc = could not find container \"e3cd9afbeab656a32aabfa4260a549dcf8a0d3d299d6eefe8a0b56c4ee148b11\": container with ID starting with e3cd9afbeab656a32aabfa4260a549dcf8a0d3d299d6eefe8a0b56c4ee148b11 not found: ID does not exist"
Oct 30 22:19:31.137525 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:19:31.137493    6133 scope.go:122] "RemoveContainer" containerID="59f545d1f37628359be89fe98525303f1090c1ff19041c990d0308102edfd5b3"
Oct 30 22:19:31.137749 ip-10-0-143-246 kubenswrapper[6133]: E1030 22:19:31.137731    6133 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"59f545d1f37628359be89fe98525303f1090c1ff19041c990d0308102edfd5b3\": container with ID starting with 59f545d1f37628359be89fe98525303f1090c1ff19041c990d0308102edfd5b3 not found: ID does not exist" containerID="59f545d1f37628359be89fe98525303f1090c1ff19041c990d0308102edfd5b3"
Oct 30 22:19:31.137792 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:19:31.137755    6133 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"59f545d1f37628359be89fe98525303f1090c1ff19041c990d0308102edfd5b3"} err="failed to get container status \"59f545d1f37628359be89fe98525303f1090c1ff19041c990d0308102edfd5b3\": rpc error: code = NotFound desc = could not find container \"59f545d1f37628359be89fe98525303f1090c1ff19041c990d0308102edfd5b3\": container with ID starting with 59f545d1f37628359be89fe98525303f1090c1ff19041c990d0308102edfd5b3 not found: ID does not exist"
Oct 30 22:19:31.137792 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:19:31.137771    6133 scope.go:122] "RemoveContainer" containerID="53028db89edb071241883cd39fba0e1db7e0d9d16dbbc9f1a1712be0dcd23e70"
Oct 30 22:19:31.138016 ip-10-0-143-246 kubenswrapper[6133]: E1030 22:19:31.137975    6133 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"53028db89edb071241883cd39fba0e1db7e0d9d16dbbc9f1a1712be0dcd23e70\": container with ID starting with 53028db89edb071241883cd39fba0e1db7e0d9d16dbbc9f1a1712be0dcd23e70 not found: ID does not exist" containerID="53028db89edb071241883cd39fba0e1db7e0d9d16dbbc9f1a1712be0dcd23e70"
Oct 30 22:19:31.138111 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:19:31.138016    6133 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"53028db89edb071241883cd39fba0e1db7e0d9d16dbbc9f1a1712be0dcd23e70"} err="failed to get container status \"53028db89edb071241883cd39fba0e1db7e0d9d16dbbc9f1a1712be0dcd23e70\": rpc error: code = NotFound desc = could not find container \"53028db89edb071241883cd39fba0e1db7e0d9d16dbbc9f1a1712be0dcd23e70\": container with ID starting with 53028db89edb071241883cd39fba0e1db7e0d9d16dbbc9f1a1712be0dcd23e70 not found: ID does not exist"
Oct 30 22:19:31.180914 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:19:31.180878    6133 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw1-2e5a75ca98/isvc-canary-create-predictor-57d466f945-cc577"]
Oct 30 22:19:31.193258 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:19:31.193230    6133 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-gw1-2e5a75ca98/isvc-canary-create-predictor-57d466f945-cc577"]
Oct 30 22:19:31.355048 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:19:31.355008    6133 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5056aa18-e249-4a78-be47-cc8df47f7920" path="/var/lib/kubelet/pods/5056aa18-e249-4a78-be47-cc8df47f7920/volumes"
Oct 30 22:19:32.801279 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:19:32.801238    6133 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-gw1-2e5a75ca98/isvc-canary-promote-v2-predictor-f9569456-88gj7"]
Oct 30 22:19:32.801693 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:19:32.801532    6133 cpu_manager.go:414] "containerMap: removing container" podUID="5056aa18-e249-4a78-be47-cc8df47f7920" containerName="storage-initializer"
Oct 30 22:19:32.801693 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:19:32.801545    6133 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5056aa18-e249-4a78-be47-cc8df47f7920" containerName="storage-initializer"
Oct 30 22:19:32.801693 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:19:32.801555    6133 cpu_manager.go:414] "containerMap: removing container" podUID="5056aa18-e249-4a78-be47-cc8df47f7920" containerName="kserve-container"
Oct 30 22:19:32.801693 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:19:32.801560    6133 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5056aa18-e249-4a78-be47-cc8df47f7920" containerName="kserve-container"
Oct 30 22:19:32.801693 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:19:32.801569    6133 cpu_manager.go:414] "containerMap: removing container" podUID="5056aa18-e249-4a78-be47-cc8df47f7920" containerName="kube-rbac-proxy"
Oct 30 22:19:32.801693 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:19:32.801575    6133 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5056aa18-e249-4a78-be47-cc8df47f7920" containerName="kube-rbac-proxy"
Oct 30 22:19:32.801693 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:19:32.801583    6133 cpu_manager.go:414] "containerMap: removing container" podUID="acaaf46b-acde-4a9d-b95f-9081342e5009" containerName="kube-rbac-proxy"
Oct 30 22:19:32.801693 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:19:32.801588    6133 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="acaaf46b-acde-4a9d-b95f-9081342e5009" containerName="kube-rbac-proxy"
Oct 30 22:19:32.801693 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:19:32.801608    6133 cpu_manager.go:414] "containerMap: removing container" podUID="acaaf46b-acde-4a9d-b95f-9081342e5009" containerName="kserve-container"
Oct 30 22:19:32.801693 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:19:32.801613    6133 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="acaaf46b-acde-4a9d-b95f-9081342e5009" containerName="kserve-container"
Oct 30 22:19:32.801693 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:19:32.801624    6133 cpu_manager.go:414] "containerMap: removing container" podUID="acaaf46b-acde-4a9d-b95f-9081342e5009" containerName="storage-initializer"
Oct 30 22:19:32.801693 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:19:32.801629    6133 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="acaaf46b-acde-4a9d-b95f-9081342e5009" containerName="storage-initializer"
Oct 30 22:19:32.801693 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:19:32.801672    6133 memory_manager.go:365] "RemoveStaleState removing state" podUID="5056aa18-e249-4a78-be47-cc8df47f7920" containerName="kserve-container"
Oct 30 22:19:32.801693 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:19:32.801680    6133 memory_manager.go:365] "RemoveStaleState removing state" podUID="5056aa18-e249-4a78-be47-cc8df47f7920" containerName="kube-rbac-proxy"
Oct 30 22:19:32.801693 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:19:32.801686    6133 memory_manager.go:365] "RemoveStaleState removing state" podUID="acaaf46b-acde-4a9d-b95f-9081342e5009" containerName="kube-rbac-proxy"
Oct 30 22:19:32.801693 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:19:32.801694    6133 memory_manager.go:365] "RemoveStaleState removing state" podUID="acaaf46b-acde-4a9d-b95f-9081342e5009" containerName="kserve-container"
Oct 30 22:19:32.806208 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:19:32.806191    6133 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-2e5a75ca98/isvc-canary-promote-v2-predictor-f9569456-88gj7"
Oct 30 22:19:32.841590 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:19:32.841561    6133 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw1-2e5a75ca98\"/\"seaweedfs-s3-creds\""
Oct 30 22:19:32.841859 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:19:32.841833    6133 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw1-2e5a75ca98\"/\"default-dockercfg-d9bw2\""
Oct 30 22:19:32.841932 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:19:32.841888    6133 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw1-2e5a75ca98\"/\"openshift-service-ca.crt\""
Oct 30 22:19:32.841932 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:19:32.841925    6133 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw1-2e5a75ca98\"/\"isvc-canary-promote-kube-rbac-proxy-sar-config\""
Oct 30 22:19:32.842039 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:19:32.841931    6133 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw1-2e5a75ca98\"/\"isvc-canary-promote-v2-predictor-serving-cert\""
Oct 30 22:19:32.842039 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:19:32.841946    6133 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw1-2e5a75ca98\"/\"kube-root-ca.crt\""
Oct 30 22:19:32.842039 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:19:32.842006    6133 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw1-2e5a75ca98\"/\"odh-kserve-custom-ca-bundle\""
Oct 30 22:19:32.855047 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:19:32.855017    6133 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw1-2e5a75ca98/isvc-canary-promote-v2-predictor-f9569456-88gj7"]
Oct 30 22:19:32.904421 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:19:32.904389    6133 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-gw1-2e5a75ca98/isvc-canary-promote-predictor-7f778847bc-5h5kp"]
Oct 30 22:19:32.907924 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:19:32.907907    6133 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-2e5a75ca98/isvc-canary-promote-predictor-7f778847bc-5h5kp"
Oct 30 22:19:32.930493 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:19:32.930465    6133 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw1-2e5a75ca98/isvc-canary-promote-predictor-7f778847bc-5h5kp"]
Oct 30 22:19:32.931794 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:19:32.931774    6133 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw1-2e5a75ca98\"/\"isvc-canary-promote-predictor-serving-cert\""
Oct 30 22:19:32.965541 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:19:32.965509    6133 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t6gxm\" (UniqueName: \"kubernetes.io/projected/e9c5e05a-b8a3-44e8-a54b-31afc28eeadf-kube-api-access-t6gxm\") pod \"isvc-canary-promote-v2-predictor-f9569456-88gj7\" (UID: \"e9c5e05a-b8a3-44e8-a54b-31afc28eeadf\") " pod="e2e-gw1-2e5a75ca98/isvc-canary-promote-v2-predictor-f9569456-88gj7"
Oct 30 22:19:32.965698 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:19:32.965563    6133 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/e9c5e05a-b8a3-44e8-a54b-31afc28eeadf-kube-rbac-proxy-sar-config\") pod \"isvc-canary-promote-v2-predictor-f9569456-88gj7\" (UID: \"e9c5e05a-b8a3-44e8-a54b-31afc28eeadf\") " pod="e2e-gw1-2e5a75ca98/isvc-canary-promote-v2-predictor-f9569456-88gj7"
Oct 30 22:19:32.965698 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:19:32.965615    6133 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/e9c5e05a-b8a3-44e8-a54b-31afc28eeadf-cabundle-cert\") pod \"isvc-canary-promote-v2-predictor-f9569456-88gj7\" (UID: \"e9c5e05a-b8a3-44e8-a54b-31afc28eeadf\") " pod="e2e-gw1-2e5a75ca98/isvc-canary-promote-v2-predictor-f9569456-88gj7"
Oct 30 22:19:32.965698 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:19:32.965664    6133 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/e9c5e05a-b8a3-44e8-a54b-31afc28eeadf-proxy-tls\") pod \"isvc-canary-promote-v2-predictor-f9569456-88gj7\" (UID: \"e9c5e05a-b8a3-44e8-a54b-31afc28eeadf\") " pod="e2e-gw1-2e5a75ca98/isvc-canary-promote-v2-predictor-f9569456-88gj7"
Oct 30 22:19:32.965825 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:19:32.965743    6133 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/e9c5e05a-b8a3-44e8-a54b-31afc28eeadf-kserve-provision-location\") pod \"isvc-canary-promote-v2-predictor-f9569456-88gj7\" (UID: \"e9c5e05a-b8a3-44e8-a54b-31afc28eeadf\") " pod="e2e-gw1-2e5a75ca98/isvc-canary-promote-v2-predictor-f9569456-88gj7"
Oct 30 22:19:33.067177 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:19:33.067067    6133 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/e9c5e05a-b8a3-44e8-a54b-31afc28eeadf-kserve-provision-location\") pod \"isvc-canary-promote-v2-predictor-f9569456-88gj7\" (UID: \"e9c5e05a-b8a3-44e8-a54b-31afc28eeadf\") " pod="e2e-gw1-2e5a75ca98/isvc-canary-promote-v2-predictor-f9569456-88gj7"
Oct 30 22:19:33.067177 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:19:33.067133    6133 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/e481d3dd-8c84-4bd0-a2dd-2bafd54b142d-kserve-provision-location\") pod \"isvc-canary-promote-predictor-7f778847bc-5h5kp\" (UID: \"e481d3dd-8c84-4bd0-a2dd-2bafd54b142d\") " pod="e2e-gw1-2e5a75ca98/isvc-canary-promote-predictor-7f778847bc-5h5kp"
Oct 30 22:19:33.067445 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:19:33.067342    6133 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t6gxm\" (UniqueName: \"kubernetes.io/projected/e9c5e05a-b8a3-44e8-a54b-31afc28eeadf-kube-api-access-t6gxm\") pod \"isvc-canary-promote-v2-predictor-f9569456-88gj7\" (UID: \"e9c5e05a-b8a3-44e8-a54b-31afc28eeadf\") " pod="e2e-gw1-2e5a75ca98/isvc-canary-promote-v2-predictor-f9569456-88gj7"
Oct 30 22:19:33.067508 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:19:33.067435    6133 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/e481d3dd-8c84-4bd0-a2dd-2bafd54b142d-kube-rbac-proxy-sar-config\") pod \"isvc-canary-promote-predictor-7f778847bc-5h5kp\" (UID: \"e481d3dd-8c84-4bd0-a2dd-2bafd54b142d\") " pod="e2e-gw1-2e5a75ca98/isvc-canary-promote-predictor-7f778847bc-5h5kp"
Oct 30 22:19:33.067508 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:19:33.067497    6133 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/e481d3dd-8c84-4bd0-a2dd-2bafd54b142d-cabundle-cert\") pod \"isvc-canary-promote-predictor-7f778847bc-5h5kp\" (UID: \"e481d3dd-8c84-4bd0-a2dd-2bafd54b142d\") " pod="e2e-gw1-2e5a75ca98/isvc-canary-promote-predictor-7f778847bc-5h5kp"
Oct 30 22:19:33.067627 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:19:33.067603    6133 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/e481d3dd-8c84-4bd0-a2dd-2bafd54b142d-proxy-tls\") pod \"isvc-canary-promote-predictor-7f778847bc-5h5kp\" (UID: \"e481d3dd-8c84-4bd0-a2dd-2bafd54b142d\") " pod="e2e-gw1-2e5a75ca98/isvc-canary-promote-predictor-7f778847bc-5h5kp"
Oct 30 22:19:33.067690 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:19:33.067639    6133 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-br2tj\" (UniqueName: \"kubernetes.io/projected/e481d3dd-8c84-4bd0-a2dd-2bafd54b142d-kube-api-access-br2tj\") pod \"isvc-canary-promote-predictor-7f778847bc-5h5kp\" (UID: \"e481d3dd-8c84-4bd0-a2dd-2bafd54b142d\") " pod="e2e-gw1-2e5a75ca98/isvc-canary-promote-predictor-7f778847bc-5h5kp"
Oct 30 22:19:33.067690 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:19:33.067671    6133 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/e9c5e05a-b8a3-44e8-a54b-31afc28eeadf-kserve-provision-location\") pod \"isvc-canary-promote-v2-predictor-f9569456-88gj7\" (UID: \"e9c5e05a-b8a3-44e8-a54b-31afc28eeadf\") " pod="e2e-gw1-2e5a75ca98/isvc-canary-promote-v2-predictor-f9569456-88gj7"
Oct 30 22:19:33.067827 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:19:33.067786    6133 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/e9c5e05a-b8a3-44e8-a54b-31afc28eeadf-kube-rbac-proxy-sar-config\") pod \"isvc-canary-promote-v2-predictor-f9569456-88gj7\" (UID: \"e9c5e05a-b8a3-44e8-a54b-31afc28eeadf\") " pod="e2e-gw1-2e5a75ca98/isvc-canary-promote-v2-predictor-f9569456-88gj7"
Oct 30 22:19:33.067888 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:19:33.067832    6133 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/e9c5e05a-b8a3-44e8-a54b-31afc28eeadf-cabundle-cert\") pod \"isvc-canary-promote-v2-predictor-f9569456-88gj7\" (UID: \"e9c5e05a-b8a3-44e8-a54b-31afc28eeadf\") " pod="e2e-gw1-2e5a75ca98/isvc-canary-promote-v2-predictor-f9569456-88gj7"
Oct 30 22:19:33.067888 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:19:33.067874    6133 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/e9c5e05a-b8a3-44e8-a54b-31afc28eeadf-proxy-tls\") pod \"isvc-canary-promote-v2-predictor-f9569456-88gj7\" (UID: \"e9c5e05a-b8a3-44e8-a54b-31afc28eeadf\") " pod="e2e-gw1-2e5a75ca98/isvc-canary-promote-v2-predictor-f9569456-88gj7"
Oct 30 22:19:33.068488 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:19:33.068461    6133 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/e9c5e05a-b8a3-44e8-a54b-31afc28eeadf-kube-rbac-proxy-sar-config\") pod \"isvc-canary-promote-v2-predictor-f9569456-88gj7\" (UID: \"e9c5e05a-b8a3-44e8-a54b-31afc28eeadf\") " pod="e2e-gw1-2e5a75ca98/isvc-canary-promote-v2-predictor-f9569456-88gj7"
Oct 30 22:19:33.068567 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:19:33.068535    6133 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/e9c5e05a-b8a3-44e8-a54b-31afc28eeadf-cabundle-cert\") pod \"isvc-canary-promote-v2-predictor-f9569456-88gj7\" (UID: \"e9c5e05a-b8a3-44e8-a54b-31afc28eeadf\") " pod="e2e-gw1-2e5a75ca98/isvc-canary-promote-v2-predictor-f9569456-88gj7"
Oct 30 22:19:33.070435 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:19:33.070412    6133 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/e9c5e05a-b8a3-44e8-a54b-31afc28eeadf-proxy-tls\") pod \"isvc-canary-promote-v2-predictor-f9569456-88gj7\" (UID: \"e9c5e05a-b8a3-44e8-a54b-31afc28eeadf\") " pod="e2e-gw1-2e5a75ca98/isvc-canary-promote-v2-predictor-f9569456-88gj7"
Oct 30 22:19:33.088940 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:19:33.088910    6133 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t6gxm\" (UniqueName: \"kubernetes.io/projected/e9c5e05a-b8a3-44e8-a54b-31afc28eeadf-kube-api-access-t6gxm\") pod \"isvc-canary-promote-v2-predictor-f9569456-88gj7\" (UID: \"e9c5e05a-b8a3-44e8-a54b-31afc28eeadf\") " pod="e2e-gw1-2e5a75ca98/isvc-canary-promote-v2-predictor-f9569456-88gj7"
Oct 30 22:19:33.122458 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:19:33.122428    6133 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-2e5a75ca98/isvc-canary-promote-v2-predictor-f9569456-88gj7"
Oct 30 22:19:33.168740 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:19:33.168709    6133 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/e481d3dd-8c84-4bd0-a2dd-2bafd54b142d-kserve-provision-location\") pod \"isvc-canary-promote-predictor-7f778847bc-5h5kp\" (UID: \"e481d3dd-8c84-4bd0-a2dd-2bafd54b142d\") " pod="e2e-gw1-2e5a75ca98/isvc-canary-promote-predictor-7f778847bc-5h5kp"
Oct 30 22:19:33.168887 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:19:33.168775    6133 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/e481d3dd-8c84-4bd0-a2dd-2bafd54b142d-kube-rbac-proxy-sar-config\") pod \"isvc-canary-promote-predictor-7f778847bc-5h5kp\" (UID: \"e481d3dd-8c84-4bd0-a2dd-2bafd54b142d\") " pod="e2e-gw1-2e5a75ca98/isvc-canary-promote-predictor-7f778847bc-5h5kp"
Oct 30 22:19:33.168887 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:19:33.168800    6133 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/e481d3dd-8c84-4bd0-a2dd-2bafd54b142d-cabundle-cert\") pod \"isvc-canary-promote-predictor-7f778847bc-5h5kp\" (UID: \"e481d3dd-8c84-4bd0-a2dd-2bafd54b142d\") " pod="e2e-gw1-2e5a75ca98/isvc-canary-promote-predictor-7f778847bc-5h5kp"
Oct 30 22:19:33.168887 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:19:33.168819    6133 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/e481d3dd-8c84-4bd0-a2dd-2bafd54b142d-proxy-tls\") pod \"isvc-canary-promote-predictor-7f778847bc-5h5kp\" (UID: \"e481d3dd-8c84-4bd0-a2dd-2bafd54b142d\") " pod="e2e-gw1-2e5a75ca98/isvc-canary-promote-predictor-7f778847bc-5h5kp"
Oct 30 22:19:33.168887 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:19:33.168840    6133 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-br2tj\" (UniqueName: \"kubernetes.io/projected/e481d3dd-8c84-4bd0-a2dd-2bafd54b142d-kube-api-access-br2tj\") pod \"isvc-canary-promote-predictor-7f778847bc-5h5kp\" (UID: \"e481d3dd-8c84-4bd0-a2dd-2bafd54b142d\") " pod="e2e-gw1-2e5a75ca98/isvc-canary-promote-predictor-7f778847bc-5h5kp"
Oct 30 22:19:33.169640 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:19:33.169179    6133 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/e481d3dd-8c84-4bd0-a2dd-2bafd54b142d-kserve-provision-location\") pod \"isvc-canary-promote-predictor-7f778847bc-5h5kp\" (UID: \"e481d3dd-8c84-4bd0-a2dd-2bafd54b142d\") " pod="e2e-gw1-2e5a75ca98/isvc-canary-promote-predictor-7f778847bc-5h5kp"
Oct 30 22:19:33.169706 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:19:33.169672    6133 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/e481d3dd-8c84-4bd0-a2dd-2bafd54b142d-kube-rbac-proxy-sar-config\") pod \"isvc-canary-promote-predictor-7f778847bc-5h5kp\" (UID: \"e481d3dd-8c84-4bd0-a2dd-2bafd54b142d\") " pod="e2e-gw1-2e5a75ca98/isvc-canary-promote-predictor-7f778847bc-5h5kp"
Oct 30 22:19:33.169890 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:19:33.169861    6133 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/e481d3dd-8c84-4bd0-a2dd-2bafd54b142d-cabundle-cert\") pod \"isvc-canary-promote-predictor-7f778847bc-5h5kp\" (UID: \"e481d3dd-8c84-4bd0-a2dd-2bafd54b142d\") " pod="e2e-gw1-2e5a75ca98/isvc-canary-promote-predictor-7f778847bc-5h5kp"
Oct 30 22:19:33.172042 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:19:33.171972    6133 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/e481d3dd-8c84-4bd0-a2dd-2bafd54b142d-proxy-tls\") pod \"isvc-canary-promote-predictor-7f778847bc-5h5kp\" (UID: \"e481d3dd-8c84-4bd0-a2dd-2bafd54b142d\") " pod="e2e-gw1-2e5a75ca98/isvc-canary-promote-predictor-7f778847bc-5h5kp"
Oct 30 22:19:33.190164 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:19:33.190136    6133 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-br2tj\" (UniqueName: \"kubernetes.io/projected/e481d3dd-8c84-4bd0-a2dd-2bafd54b142d-kube-api-access-br2tj\") pod \"isvc-canary-promote-predictor-7f778847bc-5h5kp\" (UID: \"e481d3dd-8c84-4bd0-a2dd-2bafd54b142d\") " pod="e2e-gw1-2e5a75ca98/isvc-canary-promote-predictor-7f778847bc-5h5kp"
Oct 30 22:19:33.225208 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:19:33.225172    6133 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-2e5a75ca98/isvc-canary-promote-predictor-7f778847bc-5h5kp"
Oct 30 22:19:33.429471 ip-10-0-143-246 kubenswrapper[6133]: W1030 22:19:33.429407    6133 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode9c5e05a_b8a3_44e8_a54b_31afc28eeadf.slice/crio-b8a2e3431025ca1090e2b8ae73fd8d986a9004bebb85c31cc86966458a8ca738 WatchSource:0}: Error finding container b8a2e3431025ca1090e2b8ae73fd8d986a9004bebb85c31cc86966458a8ca738: Status 404 returned error can't find the container with id b8a2e3431025ca1090e2b8ae73fd8d986a9004bebb85c31cc86966458a8ca738
Oct 30 22:19:33.430681 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:19:33.430659    6133 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw1-2e5a75ca98/isvc-canary-promote-v2-predictor-f9569456-88gj7"]
Oct 30 22:19:33.520954 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:19:33.520928    6133 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw1-2e5a75ca98/isvc-canary-promote-predictor-7f778847bc-5h5kp"]
Oct 30 22:19:33.522894 ip-10-0-143-246 kubenswrapper[6133]: W1030 22:19:33.522864    6133 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode481d3dd_8c84_4bd0_a2dd_2bafd54b142d.slice/crio-ca1fc4a4985d748e4422c99a05c76964079b7535278aa2415baffa65bd888dbd WatchSource:0}: Error finding container ca1fc4a4985d748e4422c99a05c76964079b7535278aa2415baffa65bd888dbd: Status 404 returned error can't find the container with id ca1fc4a4985d748e4422c99a05c76964079b7535278aa2415baffa65bd888dbd
Oct 30 22:19:34.126331 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:19:34.126294    6133 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-2e5a75ca98/isvc-canary-promote-v2-predictor-f9569456-88gj7" event={"ID":"e9c5e05a-b8a3-44e8-a54b-31afc28eeadf","Type":"ContainerStarted","Data":"510401dc9d26a915bb4cf8246261789f5e1063c5282bc1217a247f58f00da2ff"}
Oct 30 22:19:34.126790 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:19:34.126363    6133 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-2e5a75ca98/isvc-canary-promote-v2-predictor-f9569456-88gj7" event={"ID":"e9c5e05a-b8a3-44e8-a54b-31afc28eeadf","Type":"ContainerStarted","Data":"b8a2e3431025ca1090e2b8ae73fd8d986a9004bebb85c31cc86966458a8ca738"}
Oct 30 22:19:34.128029 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:19:34.127979    6133 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-2e5a75ca98/isvc-canary-promote-predictor-7f778847bc-5h5kp" event={"ID":"e481d3dd-8c84-4bd0-a2dd-2bafd54b142d","Type":"ContainerStarted","Data":"17c2ebd8dda53773481d05b2653aeeec1ed57a68bc6ac54ce93b01f9e740cb53"}
Oct 30 22:19:34.128148 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:19:34.128057    6133 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-2e5a75ca98/isvc-canary-promote-predictor-7f778847bc-5h5kp" event={"ID":"e481d3dd-8c84-4bd0-a2dd-2bafd54b142d","Type":"ContainerStarted","Data":"ca1fc4a4985d748e4422c99a05c76964079b7535278aa2415baffa65bd888dbd"}
Oct 30 22:19:38.150416 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:19:38.150380    6133 generic.go:356] "Generic (PLEG): container finished" podID="e481d3dd-8c84-4bd0-a2dd-2bafd54b142d" containerID="17c2ebd8dda53773481d05b2653aeeec1ed57a68bc6ac54ce93b01f9e740cb53" exitCode=0
Oct 30 22:19:38.150928 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:19:38.150455    6133 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-2e5a75ca98/isvc-canary-promote-predictor-7f778847bc-5h5kp" event={"ID":"e481d3dd-8c84-4bd0-a2dd-2bafd54b142d","Type":"ContainerDied","Data":"17c2ebd8dda53773481d05b2653aeeec1ed57a68bc6ac54ce93b01f9e740cb53"}
Oct 30 22:19:38.152017 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:19:38.151979    6133 generic.go:356] "Generic (PLEG): container finished" podID="e9c5e05a-b8a3-44e8-a54b-31afc28eeadf" containerID="510401dc9d26a915bb4cf8246261789f5e1063c5282bc1217a247f58f00da2ff" exitCode=0
Oct 30 22:19:38.152131 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:19:38.152025    6133 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-2e5a75ca98/isvc-canary-promote-v2-predictor-f9569456-88gj7" event={"ID":"e9c5e05a-b8a3-44e8-a54b-31afc28eeadf","Type":"ContainerDied","Data":"510401dc9d26a915bb4cf8246261789f5e1063c5282bc1217a247f58f00da2ff"}
Oct 30 22:19:39.158017 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:19:39.157967    6133 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-2e5a75ca98/isvc-canary-promote-v2-predictor-f9569456-88gj7" event={"ID":"e9c5e05a-b8a3-44e8-a54b-31afc28eeadf","Type":"ContainerStarted","Data":"d43be6e4622efc3b394ac32b08c39803fc40e99a817b3b089314ee537465e9c5"}
Oct 30 22:19:39.158505 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:19:39.158056    6133 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-2e5a75ca98/isvc-canary-promote-v2-predictor-f9569456-88gj7" event={"ID":"e9c5e05a-b8a3-44e8-a54b-31afc28eeadf","Type":"ContainerStarted","Data":"75de1dc55f132fd56d85e3e22fbdaddd257f10d74efd634d71f6714bd98e64ac"}
Oct 30 22:19:39.158505 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:19:39.158319    6133 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw1-2e5a75ca98/isvc-canary-promote-v2-predictor-f9569456-88gj7"
Oct 30 22:19:39.158505 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:19:39.158362    6133 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw1-2e5a75ca98/isvc-canary-promote-v2-predictor-f9569456-88gj7"
Oct 30 22:19:39.159626 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:19:39.159599    6133 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-2e5a75ca98/isvc-canary-promote-v2-predictor-f9569456-88gj7" podUID="e9c5e05a-b8a3-44e8-a54b-31afc28eeadf" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.31:8080: connect: connection refused"
Oct 30 22:19:39.160221 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:19:39.160202    6133 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-2e5a75ca98/isvc-canary-promote-predictor-7f778847bc-5h5kp" event={"ID":"e481d3dd-8c84-4bd0-a2dd-2bafd54b142d","Type":"ContainerStarted","Data":"049c970d8e5a49a686827e7e9390ca27f0fcb0b331ba2fb85a01a4965c63ce63"}
Oct 30 22:19:39.160301 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:19:39.160248    6133 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-2e5a75ca98/isvc-canary-promote-predictor-7f778847bc-5h5kp" event={"ID":"e481d3dd-8c84-4bd0-a2dd-2bafd54b142d","Type":"ContainerStarted","Data":"bc192705c7cca1f00a5f9e750d0d17fcf3b89df15861e52fec679f350e2cd554"}
Oct 30 22:19:39.160446 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:19:39.160428    6133 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw1-2e5a75ca98/isvc-canary-promote-predictor-7f778847bc-5h5kp"
Oct 30 22:19:39.160485 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:19:39.160479    6133 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw1-2e5a75ca98/isvc-canary-promote-predictor-7f778847bc-5h5kp"
Oct 30 22:19:39.161582 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:19:39.161556    6133 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-2e5a75ca98/isvc-canary-promote-predictor-7f778847bc-5h5kp" podUID="e481d3dd-8c84-4bd0-a2dd-2bafd54b142d" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.32:8080: connect: connection refused"
Oct 30 22:19:39.221244 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:19:39.221191    6133 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-gw1-2e5a75ca98/isvc-canary-promote-v2-predictor-f9569456-88gj7" podStartSLOduration=3.410637529 podStartE2EDuration="7.221178288s" totalImagesPullingTime="0s" totalInitContainerRuntime="3.810540759s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:19:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:19:39.220702114 +0000 UTC m=+1485.981458002" watchObservedRunningTime="2026-10-30 22:19:39.221178288 +0000 UTC m=+1485.981934167"
Oct 30 22:19:39.272768 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:19:39.272706    6133 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-gw1-2e5a75ca98/isvc-canary-promote-predictor-7f778847bc-5h5kp" podStartSLOduration=3.502747918 podStartE2EDuration="7.272689825s" totalImagesPullingTime="0s" totalInitContainerRuntime="3.769941907s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:19:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:19:39.271919921 +0000 UTC m=+1486.032675810" watchObservedRunningTime="2026-10-30 22:19:39.272689825 +0000 UTC m=+1486.033445713"
Oct 30 22:19:39.526780 ip-10-0-143-246 kubenswrapper[6133]: E1030 22:19:39.526740    6133 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6jppj" containerName="extractor"
Oct 30 22:19:40.164180 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:19:40.164129    6133 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-2e5a75ca98/isvc-canary-promote-predictor-7f778847bc-5h5kp" podUID="e481d3dd-8c84-4bd0-a2dd-2bafd54b142d" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.32:8080: connect: connection refused"
Oct 30 22:19:40.164602 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:19:40.164214    6133 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-2e5a75ca98/isvc-canary-promote-v2-predictor-f9569456-88gj7" podUID="e9c5e05a-b8a3-44e8-a54b-31afc28eeadf" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.31:8080: connect: connection refused"
Oct 30 22:19:45.169194 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:19:45.169151    6133 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw1-2e5a75ca98/isvc-canary-promote-v2-predictor-f9569456-88gj7"
Oct 30 22:19:45.169696 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:19:45.169669    6133 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw1-2e5a75ca98/isvc-canary-promote-predictor-7f778847bc-5h5kp"
Oct 30 22:19:45.169777 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:19:45.169718    6133 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-2e5a75ca98/isvc-canary-promote-v2-predictor-f9569456-88gj7" podUID="e9c5e05a-b8a3-44e8-a54b-31afc28eeadf" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.31:8080: connect: connection refused"
Oct 30 22:19:45.170169 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:19:45.170144    6133 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-2e5a75ca98/isvc-canary-promote-predictor-7f778847bc-5h5kp" podUID="e481d3dd-8c84-4bd0-a2dd-2bafd54b142d" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.32:8080: connect: connection refused"
Oct 30 22:19:55.169910 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:19:55.169870    6133 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-2e5a75ca98/isvc-canary-promote-v2-predictor-f9569456-88gj7" podUID="e9c5e05a-b8a3-44e8-a54b-31afc28eeadf" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.31:8080: connect: connection refused"
Oct 30 22:19:55.170405 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:19:55.170251    6133 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-2e5a75ca98/isvc-canary-promote-predictor-7f778847bc-5h5kp" podUID="e481d3dd-8c84-4bd0-a2dd-2bafd54b142d" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.32:8080: connect: connection refused"
Oct 30 22:20:05.170499 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:20:05.170450    6133 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-2e5a75ca98/isvc-canary-promote-v2-predictor-f9569456-88gj7" podUID="e9c5e05a-b8a3-44e8-a54b-31afc28eeadf" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.31:8080: connect: connection refused"
Oct 30 22:20:05.170919 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:20:05.170511    6133 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-2e5a75ca98/isvc-canary-promote-predictor-7f778847bc-5h5kp" podUID="e481d3dd-8c84-4bd0-a2dd-2bafd54b142d" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.32:8080: connect: connection refused"
Oct 30 22:20:09.526866 ip-10-0-143-246 kubenswrapper[6133]: E1030 22:20:09.526830    6133 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6jppj" containerName="extractor"
Oct 30 22:20:15.170492 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:20:15.170450    6133 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-2e5a75ca98/isvc-canary-promote-predictor-7f778847bc-5h5kp" podUID="e481d3dd-8c84-4bd0-a2dd-2bafd54b142d" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.32:8080: connect: connection refused"
Oct 30 22:20:15.171008 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:20:15.170460    6133 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-2e5a75ca98/isvc-canary-promote-v2-predictor-f9569456-88gj7" podUID="e9c5e05a-b8a3-44e8-a54b-31afc28eeadf" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.31:8080: connect: connection refused"
Oct 30 22:20:25.169836 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:20:25.169796    6133 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-2e5a75ca98/isvc-canary-promote-v2-predictor-f9569456-88gj7" podUID="e9c5e05a-b8a3-44e8-a54b-31afc28eeadf" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.31:8080: connect: connection refused"
Oct 30 22:20:25.170348 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:20:25.170315    6133 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-2e5a75ca98/isvc-canary-promote-predictor-7f778847bc-5h5kp" podUID="e481d3dd-8c84-4bd0-a2dd-2bafd54b142d" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.32:8080: connect: connection refused"
Oct 30 22:20:35.170624 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:20:35.170529    6133 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-2e5a75ca98/isvc-canary-promote-v2-predictor-f9569456-88gj7" podUID="e9c5e05a-b8a3-44e8-a54b-31afc28eeadf" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.31:8080: connect: connection refused"
Oct 30 22:20:35.171151 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:20:35.170529    6133 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-2e5a75ca98/isvc-canary-promote-predictor-7f778847bc-5h5kp" podUID="e481d3dd-8c84-4bd0-a2dd-2bafd54b142d" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.32:8080: connect: connection refused"
Oct 30 22:20:39.529218 ip-10-0-143-246 kubenswrapper[6133]: E1030 22:20:39.529170    6133 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6jppj" containerName="extractor"
Oct 30 22:20:45.170239 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:20:45.170197    6133 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-2e5a75ca98/isvc-canary-promote-predictor-7f778847bc-5h5kp" podUID="e481d3dd-8c84-4bd0-a2dd-2bafd54b142d" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.32:8080: connect: connection refused"
Oct 30 22:20:45.170716 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:20:45.170199    6133 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-2e5a75ca98/isvc-canary-promote-v2-predictor-f9569456-88gj7" podUID="e9c5e05a-b8a3-44e8-a54b-31afc28eeadf" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.31:8080: connect: connection refused"
Oct 30 22:20:49.351535 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:20:49.351492    6133 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-2e5a75ca98/isvc-canary-promote-v2-predictor-f9569456-88gj7" podUID="e9c5e05a-b8a3-44e8-a54b-31afc28eeadf" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.31:8080: connect: connection refused"
Oct 30 22:20:55.170539 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:20:55.170499    6133 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-2e5a75ca98/isvc-canary-promote-predictor-7f778847bc-5h5kp" podUID="e481d3dd-8c84-4bd0-a2dd-2bafd54b142d" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.32:8080: connect: connection refused"
Oct 30 22:20:59.354914 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:20:59.354884    6133 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw1-2e5a75ca98/isvc-canary-promote-v2-predictor-f9569456-88gj7"
Oct 30 22:20:59.770349 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:20:59.770316    6133 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw1-2e5a75ca98/isvc-canary-promote-predictor-7f778847bc-5h5kp"]
Oct 30 22:20:59.770633 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:20:59.770595    6133 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw1-2e5a75ca98/isvc-canary-promote-predictor-7f778847bc-5h5kp" podUID="e481d3dd-8c84-4bd0-a2dd-2bafd54b142d" containerName="kserve-container" containerID="cri-o://bc192705c7cca1f00a5f9e750d0d17fcf3b89df15861e52fec679f350e2cd554" gracePeriod=30
Oct 30 22:20:59.770770 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:20:59.770636    6133 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw1-2e5a75ca98/isvc-canary-promote-predictor-7f778847bc-5h5kp" podUID="e481d3dd-8c84-4bd0-a2dd-2bafd54b142d" containerName="kube-rbac-proxy" containerID="cri-o://049c970d8e5a49a686827e7e9390ca27f0fcb0b331ba2fb85a01a4965c63ce63" gracePeriod=30
Oct 30 22:20:59.771473 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:20:59.771130    6133 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw1-2e5a75ca98/isvc-canary-promote-predictor-7f778847bc-5h5kp"
Oct 30 22:21:00.164499 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:21:00.164388    6133 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-2e5a75ca98/isvc-canary-promote-predictor-7f778847bc-5h5kp" podUID="e481d3dd-8c84-4bd0-a2dd-2bafd54b142d" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.32:8643/healthz\": dial tcp 10.134.0.32:8643: connect: connection refused"
Oct 30 22:21:00.457917 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:21:00.457831    6133 generic.go:356] "Generic (PLEG): container finished" podID="e481d3dd-8c84-4bd0-a2dd-2bafd54b142d" containerID="049c970d8e5a49a686827e7e9390ca27f0fcb0b331ba2fb85a01a4965c63ce63" exitCode=0
Oct 30 22:21:00.457917 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:21:00.457902    6133 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-2e5a75ca98/isvc-canary-promote-predictor-7f778847bc-5h5kp" event={"ID":"e481d3dd-8c84-4bd0-a2dd-2bafd54b142d","Type":"ContainerDied","Data":"049c970d8e5a49a686827e7e9390ca27f0fcb0b331ba2fb85a01a4965c63ce63"}
Oct 30 22:21:05.021494 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:21:05.021463    6133 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-2e5a75ca98/isvc-canary-promote-predictor-7f778847bc-5h5kp"
Oct 30 22:21:05.155925 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:21:05.155831    6133 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/e481d3dd-8c84-4bd0-a2dd-2bafd54b142d-proxy-tls\" (UniqueName: \"kubernetes.io/secret/e481d3dd-8c84-4bd0-a2dd-2bafd54b142d-proxy-tls\") pod \"e481d3dd-8c84-4bd0-a2dd-2bafd54b142d\" (UID: \"e481d3dd-8c84-4bd0-a2dd-2bafd54b142d\") "
Oct 30 22:21:05.155925 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:21:05.155885    6133 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e481d3dd-8c84-4bd0-a2dd-2bafd54b142d-kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/e481d3dd-8c84-4bd0-a2dd-2bafd54b142d-kube-rbac-proxy-sar-config\") pod \"e481d3dd-8c84-4bd0-a2dd-2bafd54b142d\" (UID: \"e481d3dd-8c84-4bd0-a2dd-2bafd54b142d\") "
Oct 30 22:21:05.155925 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:21:05.155909    6133 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e481d3dd-8c84-4bd0-a2dd-2bafd54b142d-kube-api-access-br2tj\" (UniqueName: \"kubernetes.io/projected/e481d3dd-8c84-4bd0-a2dd-2bafd54b142d-kube-api-access-br2tj\") pod \"e481d3dd-8c84-4bd0-a2dd-2bafd54b142d\" (UID: \"e481d3dd-8c84-4bd0-a2dd-2bafd54b142d\") "
Oct 30 22:21:05.156242 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:21:05.155961    6133 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e481d3dd-8c84-4bd0-a2dd-2bafd54b142d-kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/e481d3dd-8c84-4bd0-a2dd-2bafd54b142d-kserve-provision-location\") pod \"e481d3dd-8c84-4bd0-a2dd-2bafd54b142d\" (UID: \"e481d3dd-8c84-4bd0-a2dd-2bafd54b142d\") "
Oct 30 22:21:05.156242 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:21:05.155978    6133 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e481d3dd-8c84-4bd0-a2dd-2bafd54b142d-cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/e481d3dd-8c84-4bd0-a2dd-2bafd54b142d-cabundle-cert\") pod \"e481d3dd-8c84-4bd0-a2dd-2bafd54b142d\" (UID: \"e481d3dd-8c84-4bd0-a2dd-2bafd54b142d\") "
Oct 30 22:21:05.156369 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:21:05.156339    6133 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e481d3dd-8c84-4bd0-a2dd-2bafd54b142d-kserve-provision-location" pod "e481d3dd-8c84-4bd0-a2dd-2bafd54b142d" (UID: "e481d3dd-8c84-4bd0-a2dd-2bafd54b142d"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:21:05.156415 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:21:05.156353    6133 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e481d3dd-8c84-4bd0-a2dd-2bafd54b142d-cabundle-cert" pod "e481d3dd-8c84-4bd0-a2dd-2bafd54b142d" (UID: "e481d3dd-8c84-4bd0-a2dd-2bafd54b142d"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:21:05.156415 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:21:05.156388    6133 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e481d3dd-8c84-4bd0-a2dd-2bafd54b142d-kube-rbac-proxy-sar-config" pod "e481d3dd-8c84-4bd0-a2dd-2bafd54b142d" (UID: "e481d3dd-8c84-4bd0-a2dd-2bafd54b142d"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:21:05.158152 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:21:05.158127    6133 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e481d3dd-8c84-4bd0-a2dd-2bafd54b142d-proxy-tls" pod "e481d3dd-8c84-4bd0-a2dd-2bafd54b142d" (UID: "e481d3dd-8c84-4bd0-a2dd-2bafd54b142d"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:21:05.158275 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:21:05.158204    6133 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e481d3dd-8c84-4bd0-a2dd-2bafd54b142d-kube-api-access-br2tj" pod "e481d3dd-8c84-4bd0-a2dd-2bafd54b142d" (UID: "e481d3dd-8c84-4bd0-a2dd-2bafd54b142d"). InnerVolumeSpecName "kube-api-access-br2tj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:21:05.256880 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:21:05.256842    6133 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/e481d3dd-8c84-4bd0-a2dd-2bafd54b142d-proxy-tls\") on node \"ip-10-0-143-246.ec2.internal\" DevicePath \"\""
Oct 30 22:21:05.256880 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:21:05.256873    6133 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/e481d3dd-8c84-4bd0-a2dd-2bafd54b142d-kube-rbac-proxy-sar-config\") on node \"ip-10-0-143-246.ec2.internal\" DevicePath \"\""
Oct 30 22:21:05.257137 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:21:05.256894    6133 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-br2tj\" (UniqueName: \"kubernetes.io/projected/e481d3dd-8c84-4bd0-a2dd-2bafd54b142d-kube-api-access-br2tj\") on node \"ip-10-0-143-246.ec2.internal\" DevicePath \"\""
Oct 30 22:21:05.257137 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:21:05.256903    6133 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/e481d3dd-8c84-4bd0-a2dd-2bafd54b142d-kserve-provision-location\") on node \"ip-10-0-143-246.ec2.internal\" DevicePath \"\""
Oct 30 22:21:05.257137 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:21:05.256911    6133 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/e481d3dd-8c84-4bd0-a2dd-2bafd54b142d-cabundle-cert\") on node \"ip-10-0-143-246.ec2.internal\" DevicePath \"\""
Oct 30 22:21:05.477358 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:21:05.477274    6133 generic.go:356] "Generic (PLEG): container finished" podID="e481d3dd-8c84-4bd0-a2dd-2bafd54b142d" containerID="bc192705c7cca1f00a5f9e750d0d17fcf3b89df15861e52fec679f350e2cd554" exitCode=0
Oct 30 22:21:05.477358 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:21:05.477341    6133 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-2e5a75ca98/isvc-canary-promote-predictor-7f778847bc-5h5kp"
Oct 30 22:21:05.477358 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:21:05.477348    6133 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-2e5a75ca98/isvc-canary-promote-predictor-7f778847bc-5h5kp" event={"ID":"e481d3dd-8c84-4bd0-a2dd-2bafd54b142d","Type":"ContainerDied","Data":"bc192705c7cca1f00a5f9e750d0d17fcf3b89df15861e52fec679f350e2cd554"}
Oct 30 22:21:05.477600 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:21:05.477407    6133 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-2e5a75ca98/isvc-canary-promote-predictor-7f778847bc-5h5kp" event={"ID":"e481d3dd-8c84-4bd0-a2dd-2bafd54b142d","Type":"ContainerDied","Data":"ca1fc4a4985d748e4422c99a05c76964079b7535278aa2415baffa65bd888dbd"}
Oct 30 22:21:05.477600 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:21:05.477437    6133 scope.go:122] "RemoveContainer" containerID="049c970d8e5a49a686827e7e9390ca27f0fcb0b331ba2fb85a01a4965c63ce63"
Oct 30 22:21:05.485317 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:21:05.485292    6133 scope.go:122] "RemoveContainer" containerID="bc192705c7cca1f00a5f9e750d0d17fcf3b89df15861e52fec679f350e2cd554"
Oct 30 22:21:05.493124 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:21:05.493101    6133 scope.go:122] "RemoveContainer" containerID="17c2ebd8dda53773481d05b2653aeeec1ed57a68bc6ac54ce93b01f9e740cb53"
Oct 30 22:21:05.500723 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:21:05.500707    6133 scope.go:122] "RemoveContainer" containerID="049c970d8e5a49a686827e7e9390ca27f0fcb0b331ba2fb85a01a4965c63ce63"
Oct 30 22:21:05.501015 ip-10-0-143-246 kubenswrapper[6133]: E1030 22:21:05.500981    6133 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"049c970d8e5a49a686827e7e9390ca27f0fcb0b331ba2fb85a01a4965c63ce63\": container with ID starting with 049c970d8e5a49a686827e7e9390ca27f0fcb0b331ba2fb85a01a4965c63ce63 not found: ID does not exist" containerID="049c970d8e5a49a686827e7e9390ca27f0fcb0b331ba2fb85a01a4965c63ce63"
Oct 30 22:21:05.501099 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:21:05.501034    6133 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"049c970d8e5a49a686827e7e9390ca27f0fcb0b331ba2fb85a01a4965c63ce63"} err="failed to get container status \"049c970d8e5a49a686827e7e9390ca27f0fcb0b331ba2fb85a01a4965c63ce63\": rpc error: code = NotFound desc = could not find container \"049c970d8e5a49a686827e7e9390ca27f0fcb0b331ba2fb85a01a4965c63ce63\": container with ID starting with 049c970d8e5a49a686827e7e9390ca27f0fcb0b331ba2fb85a01a4965c63ce63 not found: ID does not exist"
Oct 30 22:21:05.501099 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:21:05.501060    6133 scope.go:122] "RemoveContainer" containerID="bc192705c7cca1f00a5f9e750d0d17fcf3b89df15861e52fec679f350e2cd554"
Oct 30 22:21:05.501315 ip-10-0-143-246 kubenswrapper[6133]: E1030 22:21:05.501299    6133 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bc192705c7cca1f00a5f9e750d0d17fcf3b89df15861e52fec679f350e2cd554\": container with ID starting with bc192705c7cca1f00a5f9e750d0d17fcf3b89df15861e52fec679f350e2cd554 not found: ID does not exist" containerID="bc192705c7cca1f00a5f9e750d0d17fcf3b89df15861e52fec679f350e2cd554"
Oct 30 22:21:05.501355 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:21:05.501324    6133 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bc192705c7cca1f00a5f9e750d0d17fcf3b89df15861e52fec679f350e2cd554"} err="failed to get container status \"bc192705c7cca1f00a5f9e750d0d17fcf3b89df15861e52fec679f350e2cd554\": rpc error: code = NotFound desc = could not find container \"bc192705c7cca1f00a5f9e750d0d17fcf3b89df15861e52fec679f350e2cd554\": container with ID starting with bc192705c7cca1f00a5f9e750d0d17fcf3b89df15861e52fec679f350e2cd554 not found: ID does not exist"
Oct 30 22:21:05.501355 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:21:05.501340    6133 scope.go:122] "RemoveContainer" containerID="17c2ebd8dda53773481d05b2653aeeec1ed57a68bc6ac54ce93b01f9e740cb53"
Oct 30 22:21:05.501547 ip-10-0-143-246 kubenswrapper[6133]: E1030 22:21:05.501530    6133 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"17c2ebd8dda53773481d05b2653aeeec1ed57a68bc6ac54ce93b01f9e740cb53\": container with ID starting with 17c2ebd8dda53773481d05b2653aeeec1ed57a68bc6ac54ce93b01f9e740cb53 not found: ID does not exist" containerID="17c2ebd8dda53773481d05b2653aeeec1ed57a68bc6ac54ce93b01f9e740cb53"
Oct 30 22:21:05.501600 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:21:05.501555    6133 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"17c2ebd8dda53773481d05b2653aeeec1ed57a68bc6ac54ce93b01f9e740cb53"} err="failed to get container status \"17c2ebd8dda53773481d05b2653aeeec1ed57a68bc6ac54ce93b01f9e740cb53\": rpc error: code = NotFound desc = could not find container \"17c2ebd8dda53773481d05b2653aeeec1ed57a68bc6ac54ce93b01f9e740cb53\": container with ID starting with 17c2ebd8dda53773481d05b2653aeeec1ed57a68bc6ac54ce93b01f9e740cb53 not found: ID does not exist"
Oct 30 22:21:05.531946 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:21:05.531911    6133 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw1-2e5a75ca98/isvc-canary-promote-predictor-7f778847bc-5h5kp"]
Oct 30 22:21:05.543969 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:21:05.543943    6133 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-gw1-2e5a75ca98/isvc-canary-promote-predictor-7f778847bc-5h5kp"]
Oct 30 22:21:07.354801 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:21:07.354762    6133 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e481d3dd-8c84-4bd0-a2dd-2bafd54b142d" path="/var/lib/kubelet/pods/e481d3dd-8c84-4bd0-a2dd-2bafd54b142d/volumes"
Oct 30 22:21:09.527042 ip-10-0-143-246 kubenswrapper[6133]: E1030 22:21:09.527007    6133 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6jppj" containerName="extractor"
Oct 30 22:21:39.527210 ip-10-0-143-246 kubenswrapper[6133]: E1030 22:21:39.527171    6133 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6jppj" containerName="extractor"
Oct 30 22:22:09.527045 ip-10-0-143-246 kubenswrapper[6133]: E1030 22:22:09.526945    6133 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6jppj" containerName="extractor"
Oct 30 22:22:33.561090 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:22:33.561053    6133 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw1-2e5a75ca98/isvc-canary-promote-v2-predictor-f9569456-88gj7"]
Oct 30 22:22:33.561644 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:22:33.561292    6133 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw1-2e5a75ca98/isvc-canary-promote-v2-predictor-f9569456-88gj7" podUID="e9c5e05a-b8a3-44e8-a54b-31afc28eeadf" containerName="kserve-container" containerID="cri-o://75de1dc55f132fd56d85e3e22fbdaddd257f10d74efd634d71f6714bd98e64ac" gracePeriod=30
Oct 30 22:22:33.561644 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:22:33.561333    6133 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw1-2e5a75ca98/isvc-canary-promote-v2-predictor-f9569456-88gj7" podUID="e9c5e05a-b8a3-44e8-a54b-31afc28eeadf" containerName="kube-rbac-proxy" containerID="cri-o://d43be6e4622efc3b394ac32b08c39803fc40e99a817b3b089314ee537465e9c5" gracePeriod=30
Oct 30 22:22:33.803391 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:22:33.803357    6133 generic.go:356] "Generic (PLEG): container finished" podID="e9c5e05a-b8a3-44e8-a54b-31afc28eeadf" containerID="d43be6e4622efc3b394ac32b08c39803fc40e99a817b3b089314ee537465e9c5" exitCode=0
Oct 30 22:22:33.803391 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:22:33.803397    6133 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-2e5a75ca98/isvc-canary-promote-v2-predictor-f9569456-88gj7" event={"ID":"e9c5e05a-b8a3-44e8-a54b-31afc28eeadf","Type":"ContainerDied","Data":"d43be6e4622efc3b394ac32b08c39803fc40e99a817b3b089314ee537465e9c5"}
Oct 30 22:22:35.164605 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:22:35.164560    6133 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-2e5a75ca98/isvc-canary-promote-v2-predictor-f9569456-88gj7" podUID="e9c5e05a-b8a3-44e8-a54b-31afc28eeadf" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.31:8643/healthz\": dial tcp 10.134.0.31:8643: connect: connection refused"
Oct 30 22:22:38.802710 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:22:38.802680    6133 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-2e5a75ca98/isvc-canary-promote-v2-predictor-f9569456-88gj7"
Oct 30 22:22:38.824429 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:22:38.824398    6133 generic.go:356] "Generic (PLEG): container finished" podID="e9c5e05a-b8a3-44e8-a54b-31afc28eeadf" containerID="75de1dc55f132fd56d85e3e22fbdaddd257f10d74efd634d71f6714bd98e64ac" exitCode=0
Oct 30 22:22:38.824622 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:22:38.824459    6133 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-2e5a75ca98/isvc-canary-promote-v2-predictor-f9569456-88gj7"
Oct 30 22:22:38.824622 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:22:38.824473    6133 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-2e5a75ca98/isvc-canary-promote-v2-predictor-f9569456-88gj7" event={"ID":"e9c5e05a-b8a3-44e8-a54b-31afc28eeadf","Type":"ContainerDied","Data":"75de1dc55f132fd56d85e3e22fbdaddd257f10d74efd634d71f6714bd98e64ac"}
Oct 30 22:22:38.824622 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:22:38.824559    6133 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-2e5a75ca98/isvc-canary-promote-v2-predictor-f9569456-88gj7" event={"ID":"e9c5e05a-b8a3-44e8-a54b-31afc28eeadf","Type":"ContainerDied","Data":"b8a2e3431025ca1090e2b8ae73fd8d986a9004bebb85c31cc86966458a8ca738"}
Oct 30 22:22:38.824622 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:22:38.824600    6133 scope.go:122] "RemoveContainer" containerID="d43be6e4622efc3b394ac32b08c39803fc40e99a817b3b089314ee537465e9c5"
Oct 30 22:22:38.832434 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:22:38.832415    6133 scope.go:122] "RemoveContainer" containerID="75de1dc55f132fd56d85e3e22fbdaddd257f10d74efd634d71f6714bd98e64ac"
Oct 30 22:22:38.839920 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:22:38.839896    6133 scope.go:122] "RemoveContainer" containerID="510401dc9d26a915bb4cf8246261789f5e1063c5282bc1217a247f58f00da2ff"
Oct 30 22:22:38.847127 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:22:38.847102    6133 scope.go:122] "RemoveContainer" containerID="d43be6e4622efc3b394ac32b08c39803fc40e99a817b3b089314ee537465e9c5"
Oct 30 22:22:38.847397 ip-10-0-143-246 kubenswrapper[6133]: E1030 22:22:38.847376    6133 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d43be6e4622efc3b394ac32b08c39803fc40e99a817b3b089314ee537465e9c5\": container with ID starting with d43be6e4622efc3b394ac32b08c39803fc40e99a817b3b089314ee537465e9c5 not found: ID does not exist" containerID="d43be6e4622efc3b394ac32b08c39803fc40e99a817b3b089314ee537465e9c5"
Oct 30 22:22:38.847483 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:22:38.847410    6133 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d43be6e4622efc3b394ac32b08c39803fc40e99a817b3b089314ee537465e9c5"} err="failed to get container status \"d43be6e4622efc3b394ac32b08c39803fc40e99a817b3b089314ee537465e9c5\": rpc error: code = NotFound desc = could not find container \"d43be6e4622efc3b394ac32b08c39803fc40e99a817b3b089314ee537465e9c5\": container with ID starting with d43be6e4622efc3b394ac32b08c39803fc40e99a817b3b089314ee537465e9c5 not found: ID does not exist"
Oct 30 22:22:38.847483 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:22:38.847436    6133 scope.go:122] "RemoveContainer" containerID="75de1dc55f132fd56d85e3e22fbdaddd257f10d74efd634d71f6714bd98e64ac"
Oct 30 22:22:38.847706 ip-10-0-143-246 kubenswrapper[6133]: E1030 22:22:38.847690    6133 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"75de1dc55f132fd56d85e3e22fbdaddd257f10d74efd634d71f6714bd98e64ac\": container with ID starting with 75de1dc55f132fd56d85e3e22fbdaddd257f10d74efd634d71f6714bd98e64ac not found: ID does not exist" containerID="75de1dc55f132fd56d85e3e22fbdaddd257f10d74efd634d71f6714bd98e64ac"
Oct 30 22:22:38.847772 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:22:38.847718    6133 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"75de1dc55f132fd56d85e3e22fbdaddd257f10d74efd634d71f6714bd98e64ac"} err="failed to get container status \"75de1dc55f132fd56d85e3e22fbdaddd257f10d74efd634d71f6714bd98e64ac\": rpc error: code = NotFound desc = could not find container \"75de1dc55f132fd56d85e3e22fbdaddd257f10d74efd634d71f6714bd98e64ac\": container with ID starting with 75de1dc55f132fd56d85e3e22fbdaddd257f10d74efd634d71f6714bd98e64ac not found: ID does not exist"
Oct 30 22:22:38.847772 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:22:38.847738    6133 scope.go:122] "RemoveContainer" containerID="510401dc9d26a915bb4cf8246261789f5e1063c5282bc1217a247f58f00da2ff"
Oct 30 22:22:38.847969 ip-10-0-143-246 kubenswrapper[6133]: E1030 22:22:38.847949    6133 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"510401dc9d26a915bb4cf8246261789f5e1063c5282bc1217a247f58f00da2ff\": container with ID starting with 510401dc9d26a915bb4cf8246261789f5e1063c5282bc1217a247f58f00da2ff not found: ID does not exist" containerID="510401dc9d26a915bb4cf8246261789f5e1063c5282bc1217a247f58f00da2ff"
Oct 30 22:22:38.848032 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:22:38.847974    6133 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"510401dc9d26a915bb4cf8246261789f5e1063c5282bc1217a247f58f00da2ff"} err="failed to get container status \"510401dc9d26a915bb4cf8246261789f5e1063c5282bc1217a247f58f00da2ff\": rpc error: code = NotFound desc = could not find container \"510401dc9d26a915bb4cf8246261789f5e1063c5282bc1217a247f58f00da2ff\": container with ID starting with 510401dc9d26a915bb4cf8246261789f5e1063c5282bc1217a247f58f00da2ff not found: ID does not exist"
Oct 30 22:22:38.895602 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:22:38.895568    6133 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e9c5e05a-b8a3-44e8-a54b-31afc28eeadf-kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/e9c5e05a-b8a3-44e8-a54b-31afc28eeadf-kserve-provision-location\") pod \"e9c5e05a-b8a3-44e8-a54b-31afc28eeadf\" (UID: \"e9c5e05a-b8a3-44e8-a54b-31afc28eeadf\") "
Oct 30 22:22:38.895812 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:22:38.895645    6133 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e9c5e05a-b8a3-44e8-a54b-31afc28eeadf-cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/e9c5e05a-b8a3-44e8-a54b-31afc28eeadf-cabundle-cert\") pod \"e9c5e05a-b8a3-44e8-a54b-31afc28eeadf\" (UID: \"e9c5e05a-b8a3-44e8-a54b-31afc28eeadf\") "
Oct 30 22:22:38.895812 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:22:38.895667    6133 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/e9c5e05a-b8a3-44e8-a54b-31afc28eeadf-proxy-tls\" (UniqueName: \"kubernetes.io/secret/e9c5e05a-b8a3-44e8-a54b-31afc28eeadf-proxy-tls\") pod \"e9c5e05a-b8a3-44e8-a54b-31afc28eeadf\" (UID: \"e9c5e05a-b8a3-44e8-a54b-31afc28eeadf\") "
Oct 30 22:22:38.895812 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:22:38.895691    6133 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e9c5e05a-b8a3-44e8-a54b-31afc28eeadf-kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/e9c5e05a-b8a3-44e8-a54b-31afc28eeadf-kube-rbac-proxy-sar-config\") pod \"e9c5e05a-b8a3-44e8-a54b-31afc28eeadf\" (UID: \"e9c5e05a-b8a3-44e8-a54b-31afc28eeadf\") "
Oct 30 22:22:38.895812 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:22:38.895743    6133 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e9c5e05a-b8a3-44e8-a54b-31afc28eeadf-kube-api-access-t6gxm\" (UniqueName: \"kubernetes.io/projected/e9c5e05a-b8a3-44e8-a54b-31afc28eeadf-kube-api-access-t6gxm\") pod \"e9c5e05a-b8a3-44e8-a54b-31afc28eeadf\" (UID: \"e9c5e05a-b8a3-44e8-a54b-31afc28eeadf\") "
Oct 30 22:22:38.896087 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:22:38.895969    6133 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e9c5e05a-b8a3-44e8-a54b-31afc28eeadf-kserve-provision-location" pod "e9c5e05a-b8a3-44e8-a54b-31afc28eeadf" (UID: "e9c5e05a-b8a3-44e8-a54b-31afc28eeadf"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:22:38.896150 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:22:38.896089    6133 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e9c5e05a-b8a3-44e8-a54b-31afc28eeadf-kube-rbac-proxy-sar-config" pod "e9c5e05a-b8a3-44e8-a54b-31afc28eeadf" (UID: "e9c5e05a-b8a3-44e8-a54b-31afc28eeadf"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:22:38.896150 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:22:38.896116    6133 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e9c5e05a-b8a3-44e8-a54b-31afc28eeadf-cabundle-cert" pod "e9c5e05a-b8a3-44e8-a54b-31afc28eeadf" (UID: "e9c5e05a-b8a3-44e8-a54b-31afc28eeadf"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:22:38.897890 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:22:38.897862    6133 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e9c5e05a-b8a3-44e8-a54b-31afc28eeadf-proxy-tls" pod "e9c5e05a-b8a3-44e8-a54b-31afc28eeadf" (UID: "e9c5e05a-b8a3-44e8-a54b-31afc28eeadf"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:22:38.898019 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:22:38.897922    6133 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e9c5e05a-b8a3-44e8-a54b-31afc28eeadf-kube-api-access-t6gxm" pod "e9c5e05a-b8a3-44e8-a54b-31afc28eeadf" (UID: "e9c5e05a-b8a3-44e8-a54b-31afc28eeadf"). InnerVolumeSpecName "kube-api-access-t6gxm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:22:38.997344 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:22:38.997301    6133 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t6gxm\" (UniqueName: \"kubernetes.io/projected/e9c5e05a-b8a3-44e8-a54b-31afc28eeadf-kube-api-access-t6gxm\") on node \"ip-10-0-143-246.ec2.internal\" DevicePath \"\""
Oct 30 22:22:38.997344 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:22:38.997333    6133 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/e9c5e05a-b8a3-44e8-a54b-31afc28eeadf-kserve-provision-location\") on node \"ip-10-0-143-246.ec2.internal\" DevicePath \"\""
Oct 30 22:22:38.997344 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:22:38.997347    6133 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/e9c5e05a-b8a3-44e8-a54b-31afc28eeadf-cabundle-cert\") on node \"ip-10-0-143-246.ec2.internal\" DevicePath \"\""
Oct 30 22:22:38.997589 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:22:38.997358    6133 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/e9c5e05a-b8a3-44e8-a54b-31afc28eeadf-proxy-tls\") on node \"ip-10-0-143-246.ec2.internal\" DevicePath \"\""
Oct 30 22:22:38.997589 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:22:38.997366    6133 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/e9c5e05a-b8a3-44e8-a54b-31afc28eeadf-kube-rbac-proxy-sar-config\") on node \"ip-10-0-143-246.ec2.internal\" DevicePath \"\""
Oct 30 22:22:39.171166 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:22:39.171116    6133 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw1-2e5a75ca98/isvc-canary-promote-v2-predictor-f9569456-88gj7"]
Oct 30 22:22:39.183429 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:22:39.183404    6133 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-gw1-2e5a75ca98/isvc-canary-promote-v2-predictor-f9569456-88gj7"]
Oct 30 22:22:39.355686 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:22:39.355651    6133 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e9c5e05a-b8a3-44e8-a54b-31afc28eeadf" path="/var/lib/kubelet/pods/e9c5e05a-b8a3-44e8-a54b-31afc28eeadf/volumes"
Oct 30 22:22:39.528740 ip-10-0-143-246 kubenswrapper[6133]: E1030 22:22:39.528694    6133 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6jppj" containerName="extractor"
Oct 30 22:22:51.620697 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:22:51.620661    6133 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-gw0-f92f81a7ef/message-dumper-raw-cipn-predictor-57464d98d7-scpm6"]
Oct 30 22:22:51.621207 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:22:51.621152    6133 cpu_manager.go:414] "containerMap: removing container" podUID="e9c5e05a-b8a3-44e8-a54b-31afc28eeadf" containerName="kserve-container"
Oct 30 22:22:51.621207 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:22:51.621171    6133 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e9c5e05a-b8a3-44e8-a54b-31afc28eeadf" containerName="kserve-container"
Oct 30 22:22:51.621207 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:22:51.621183    6133 cpu_manager.go:414] "containerMap: removing container" podUID="e9c5e05a-b8a3-44e8-a54b-31afc28eeadf" containerName="kube-rbac-proxy"
Oct 30 22:22:51.621207 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:22:51.621190    6133 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e9c5e05a-b8a3-44e8-a54b-31afc28eeadf" containerName="kube-rbac-proxy"
Oct 30 22:22:51.621207 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:22:51.621203    6133 cpu_manager.go:414] "containerMap: removing container" podUID="e481d3dd-8c84-4bd0-a2dd-2bafd54b142d" containerName="kube-rbac-proxy"
Oct 30 22:22:51.621471 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:22:51.621213    6133 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e481d3dd-8c84-4bd0-a2dd-2bafd54b142d" containerName="kube-rbac-proxy"
Oct 30 22:22:51.621471 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:22:51.621226    6133 cpu_manager.go:414] "containerMap: removing container" podUID="e9c5e05a-b8a3-44e8-a54b-31afc28eeadf" containerName="storage-initializer"
Oct 30 22:22:51.621471 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:22:51.621234    6133 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e9c5e05a-b8a3-44e8-a54b-31afc28eeadf" containerName="storage-initializer"
Oct 30 22:22:51.621471 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:22:51.621251    6133 cpu_manager.go:414] "containerMap: removing container" podUID="e481d3dd-8c84-4bd0-a2dd-2bafd54b142d" containerName="kserve-container"
Oct 30 22:22:51.621471 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:22:51.621258    6133 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e481d3dd-8c84-4bd0-a2dd-2bafd54b142d" containerName="kserve-container"
Oct 30 22:22:51.621471 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:22:51.621274    6133 cpu_manager.go:414] "containerMap: removing container" podUID="e481d3dd-8c84-4bd0-a2dd-2bafd54b142d" containerName="storage-initializer"
Oct 30 22:22:51.621471 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:22:51.621281    6133 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e481d3dd-8c84-4bd0-a2dd-2bafd54b142d" containerName="storage-initializer"
Oct 30 22:22:51.621471 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:22:51.621369    6133 memory_manager.go:365] "RemoveStaleState removing state" podUID="e9c5e05a-b8a3-44e8-a54b-31afc28eeadf" containerName="kserve-container"
Oct 30 22:22:51.621471 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:22:51.621380    6133 memory_manager.go:365] "RemoveStaleState removing state" podUID="e9c5e05a-b8a3-44e8-a54b-31afc28eeadf" containerName="kube-rbac-proxy"
Oct 30 22:22:51.621471 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:22:51.621392    6133 memory_manager.go:365] "RemoveStaleState removing state" podUID="e481d3dd-8c84-4bd0-a2dd-2bafd54b142d" containerName="kube-rbac-proxy"
Oct 30 22:22:51.621471 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:22:51.621403    6133 memory_manager.go:365] "RemoveStaleState removing state" podUID="e481d3dd-8c84-4bd0-a2dd-2bafd54b142d" containerName="kserve-container"
Oct 30 22:22:51.624380 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:22:51.624359    6133 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-f92f81a7ef/message-dumper-raw-cipn-predictor-57464d98d7-scpm6"
Oct 30 22:22:51.640625 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:22:51.640599    6133 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw0-f92f81a7ef\"/\"openshift-service-ca.crt\""
Oct 30 22:22:51.652870 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:22:51.652843    6133 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw0-f92f81a7ef\"/\"message-dumper-raw-cipn-kube-rbac-proxy-sar-config\""
Oct 30 22:22:51.653035 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:22:51.652892    6133 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw0-f92f81a7ef\"/\"default-dockercfg-kmp4v\""
Oct 30 22:22:51.653035 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:22:51.652902    6133 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw0-f92f81a7ef\"/\"kube-root-ca.crt\""
Oct 30 22:22:51.653035 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:22:51.652945    6133 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw0-f92f81a7ef\"/\"message-dumper-raw-cipn-predictor-serving-cert\""
Oct 30 22:22:51.654229 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:22:51.654211    6133 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw0-f92f81a7ef/message-dumper-raw-cipn-predictor-57464d98d7-scpm6"]
Oct 30 22:22:51.796094 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:22:51.796063    6133 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qxqwt\" (UniqueName: \"kubernetes.io/projected/864efcb3-72df-4a51-84ec-fa24de8ab96b-kube-api-access-qxqwt\") pod \"message-dumper-raw-cipn-predictor-57464d98d7-scpm6\" (UID: \"864efcb3-72df-4a51-84ec-fa24de8ab96b\") " pod="e2e-gw0-f92f81a7ef/message-dumper-raw-cipn-predictor-57464d98d7-scpm6"
Oct 30 22:22:51.796303 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:22:51.796121    6133 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/864efcb3-72df-4a51-84ec-fa24de8ab96b-proxy-tls\") pod \"message-dumper-raw-cipn-predictor-57464d98d7-scpm6\" (UID: \"864efcb3-72df-4a51-84ec-fa24de8ab96b\") " pod="e2e-gw0-f92f81a7ef/message-dumper-raw-cipn-predictor-57464d98d7-scpm6"
Oct 30 22:22:51.796373 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:22:51.796301    6133 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/864efcb3-72df-4a51-84ec-fa24de8ab96b-kube-rbac-proxy-sar-config\") pod \"message-dumper-raw-cipn-predictor-57464d98d7-scpm6\" (UID: \"864efcb3-72df-4a51-84ec-fa24de8ab96b\") " pod="e2e-gw0-f92f81a7ef/message-dumper-raw-cipn-predictor-57464d98d7-scpm6"
Oct 30 22:22:51.897454 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:22:51.897350    6133 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qxqwt\" (UniqueName: \"kubernetes.io/projected/864efcb3-72df-4a51-84ec-fa24de8ab96b-kube-api-access-qxqwt\") pod \"message-dumper-raw-cipn-predictor-57464d98d7-scpm6\" (UID: \"864efcb3-72df-4a51-84ec-fa24de8ab96b\") " pod="e2e-gw0-f92f81a7ef/message-dumper-raw-cipn-predictor-57464d98d7-scpm6"
Oct 30 22:22:51.897454 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:22:51.897434    6133 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/864efcb3-72df-4a51-84ec-fa24de8ab96b-proxy-tls\") pod \"message-dumper-raw-cipn-predictor-57464d98d7-scpm6\" (UID: \"864efcb3-72df-4a51-84ec-fa24de8ab96b\") " pod="e2e-gw0-f92f81a7ef/message-dumper-raw-cipn-predictor-57464d98d7-scpm6"
Oct 30 22:22:51.897703 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:22:51.897501    6133 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/864efcb3-72df-4a51-84ec-fa24de8ab96b-kube-rbac-proxy-sar-config\") pod \"message-dumper-raw-cipn-predictor-57464d98d7-scpm6\" (UID: \"864efcb3-72df-4a51-84ec-fa24de8ab96b\") " pod="e2e-gw0-f92f81a7ef/message-dumper-raw-cipn-predictor-57464d98d7-scpm6"
Oct 30 22:22:51.898321 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:22:51.898293    6133 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/864efcb3-72df-4a51-84ec-fa24de8ab96b-kube-rbac-proxy-sar-config\") pod \"message-dumper-raw-cipn-predictor-57464d98d7-scpm6\" (UID: \"864efcb3-72df-4a51-84ec-fa24de8ab96b\") " pod="e2e-gw0-f92f81a7ef/message-dumper-raw-cipn-predictor-57464d98d7-scpm6"
Oct 30 22:22:51.900031 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:22:51.900005    6133 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/864efcb3-72df-4a51-84ec-fa24de8ab96b-proxy-tls\") pod \"message-dumper-raw-cipn-predictor-57464d98d7-scpm6\" (UID: \"864efcb3-72df-4a51-84ec-fa24de8ab96b\") " pod="e2e-gw0-f92f81a7ef/message-dumper-raw-cipn-predictor-57464d98d7-scpm6"
Oct 30 22:22:51.919972 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:22:51.919945    6133 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qxqwt\" (UniqueName: \"kubernetes.io/projected/864efcb3-72df-4a51-84ec-fa24de8ab96b-kube-api-access-qxqwt\") pod \"message-dumper-raw-cipn-predictor-57464d98d7-scpm6\" (UID: \"864efcb3-72df-4a51-84ec-fa24de8ab96b\") " pod="e2e-gw0-f92f81a7ef/message-dumper-raw-cipn-predictor-57464d98d7-scpm6"
Oct 30 22:22:51.938851 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:22:51.938822    6133 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-f92f81a7ef/message-dumper-raw-cipn-predictor-57464d98d7-scpm6"
Oct 30 22:22:52.233468 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:22:52.233446    6133 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw0-f92f81a7ef/message-dumper-raw-cipn-predictor-57464d98d7-scpm6"]
Oct 30 22:22:52.235832 ip-10-0-143-246 kubenswrapper[6133]: W1030 22:22:52.235804    6133 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod864efcb3_72df_4a51_84ec_fa24de8ab96b.slice/crio-f8cec0d203df39aec11d3de0179b79fb7b7442d78acdca545616630373a57cdf WatchSource:0}: Error finding container f8cec0d203df39aec11d3de0179b79fb7b7442d78acdca545616630373a57cdf: Status 404 returned error can't find the container with id f8cec0d203df39aec11d3de0179b79fb7b7442d78acdca545616630373a57cdf
Oct 30 22:22:52.874924 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:22:52.874887    6133 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-f92f81a7ef/message-dumper-raw-cipn-predictor-57464d98d7-scpm6" event={"ID":"864efcb3-72df-4a51-84ec-fa24de8ab96b","Type":"ContainerStarted","Data":"e4eac2bfa245c06bf7aee006330d161eb6274803e7595f4eb4fd1d29d304af7a"}
Oct 30 22:22:52.875450 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:22:52.874942    6133 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-f92f81a7ef/message-dumper-raw-cipn-predictor-57464d98d7-scpm6" event={"ID":"864efcb3-72df-4a51-84ec-fa24de8ab96b","Type":"ContainerStarted","Data":"883005143fd3c459fc961107352e421f7a5d163ed9c8dd7be677f1a9f1920923"}
Oct 30 22:22:52.875450 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:22:52.874962    6133 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-f92f81a7ef/message-dumper-raw-cipn-predictor-57464d98d7-scpm6" event={"ID":"864efcb3-72df-4a51-84ec-fa24de8ab96b","Type":"ContainerStarted","Data":"f8cec0d203df39aec11d3de0179b79fb7b7442d78acdca545616630373a57cdf"}
Oct 30 22:22:52.875450 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:22:52.875020    6133 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw0-f92f81a7ef/message-dumper-raw-cipn-predictor-57464d98d7-scpm6"
Oct 30 22:22:52.919300 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:22:52.919252    6133 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-gw0-f92f81a7ef/message-dumper-raw-cipn-predictor-57464d98d7-scpm6" podStartSLOduration=1.919237377 podStartE2EDuration="1.919237377s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:22:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:22:52.918181015 +0000 UTC m=+1679.678936903" watchObservedRunningTime="2026-10-30 22:22:52.919237377 +0000 UTC m=+1679.679993266"
Oct 30 22:22:53.878335 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:22:53.878246    6133 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw0-f92f81a7ef/message-dumper-raw-cipn-predictor-57464d98d7-scpm6"
Oct 30 22:22:53.880171 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:22:53.880147    6133 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw0-f92f81a7ef/message-dumper-raw-cipn-predictor-57464d98d7-scpm6"
Oct 30 22:23:00.890571 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:23:00.890542    6133 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw0-f92f81a7ef/message-dumper-raw-cipn-predictor-57464d98d7-scpm6"
Oct 30 22:23:09.526723 ip-10-0-143-246 kubenswrapper[6133]: E1030 22:23:09.526683    6133 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6jppj" containerName="extractor"
Oct 30 22:23:11.880628 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:23:11.880594    6133 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-gw0-f92f81a7ef/isvc-logger-raw-cipn-predictor-d64bbfccf-j2dsg"]
Oct 30 22:23:11.885658 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:23:11.885630    6133 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-f92f81a7ef/isvc-logger-raw-cipn-predictor-d64bbfccf-j2dsg"
Oct 30 22:23:11.901752 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:23:11.901722    6133 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw0-f92f81a7ef\"/\"seaweedfs-s3-creds\""
Oct 30 22:23:11.901975 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:23:11.901957    6133 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw0-f92f81a7ef\"/\"isvc-logger-raw-cipn-predictor-serving-cert\""
Oct 30 22:23:11.913813 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:23:11.913753    6133 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw0-f92f81a7ef\"/\"isvc-logger-raw-cipn-kube-rbac-proxy-sar-config\""
Oct 30 22:23:11.913813 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:23:11.913753    6133 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw0-f92f81a7ef\"/\"odh-kserve-custom-ca-bundle\""
Oct 30 22:23:11.914848 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:23:11.914826    6133 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw0-f92f81a7ef/isvc-logger-raw-cipn-predictor-d64bbfccf-j2dsg"]
Oct 30 22:23:11.950846 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:23:11.950807    6133 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/ebde8ce5-b706-4874-be47-578925b6509c-proxy-tls\") pod \"isvc-logger-raw-cipn-predictor-d64bbfccf-j2dsg\" (UID: \"ebde8ce5-b706-4874-be47-578925b6509c\") " pod="e2e-gw0-f92f81a7ef/isvc-logger-raw-cipn-predictor-d64bbfccf-j2dsg"
Oct 30 22:23:11.950846 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:23:11.950844    6133 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/ebde8ce5-b706-4874-be47-578925b6509c-cabundle-cert\") pod \"isvc-logger-raw-cipn-predictor-d64bbfccf-j2dsg\" (UID: \"ebde8ce5-b706-4874-be47-578925b6509c\") " pod="e2e-gw0-f92f81a7ef/isvc-logger-raw-cipn-predictor-d64bbfccf-j2dsg"
Oct 30 22:23:11.951160 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:23:11.950870    6133 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/ebde8ce5-b706-4874-be47-578925b6509c-kube-rbac-proxy-sar-config\") pod \"isvc-logger-raw-cipn-predictor-d64bbfccf-j2dsg\" (UID: \"ebde8ce5-b706-4874-be47-578925b6509c\") " pod="e2e-gw0-f92f81a7ef/isvc-logger-raw-cipn-predictor-d64bbfccf-j2dsg"
Oct 30 22:23:11.951160 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:23:11.951043    6133 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2zpcj\" (UniqueName: \"kubernetes.io/projected/ebde8ce5-b706-4874-be47-578925b6509c-kube-api-access-2zpcj\") pod \"isvc-logger-raw-cipn-predictor-d64bbfccf-j2dsg\" (UID: \"ebde8ce5-b706-4874-be47-578925b6509c\") " pod="e2e-gw0-f92f81a7ef/isvc-logger-raw-cipn-predictor-d64bbfccf-j2dsg"
Oct 30 22:23:11.951160 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:23:11.951088    6133 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/ebde8ce5-b706-4874-be47-578925b6509c-kserve-provision-location\") pod \"isvc-logger-raw-cipn-predictor-d64bbfccf-j2dsg\" (UID: \"ebde8ce5-b706-4874-be47-578925b6509c\") " pod="e2e-gw0-f92f81a7ef/isvc-logger-raw-cipn-predictor-d64bbfccf-j2dsg"
Oct 30 22:23:12.052315 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:23:12.052274    6133 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/ebde8ce5-b706-4874-be47-578925b6509c-kube-rbac-proxy-sar-config\") pod \"isvc-logger-raw-cipn-predictor-d64bbfccf-j2dsg\" (UID: \"ebde8ce5-b706-4874-be47-578925b6509c\") " pod="e2e-gw0-f92f81a7ef/isvc-logger-raw-cipn-predictor-d64bbfccf-j2dsg"
Oct 30 22:23:12.052560 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:23:12.052340    6133 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2zpcj\" (UniqueName: \"kubernetes.io/projected/ebde8ce5-b706-4874-be47-578925b6509c-kube-api-access-2zpcj\") pod \"isvc-logger-raw-cipn-predictor-d64bbfccf-j2dsg\" (UID: \"ebde8ce5-b706-4874-be47-578925b6509c\") " pod="e2e-gw0-f92f81a7ef/isvc-logger-raw-cipn-predictor-d64bbfccf-j2dsg"
Oct 30 22:23:12.052560 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:23:12.052379    6133 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/ebde8ce5-b706-4874-be47-578925b6509c-kserve-provision-location\") pod \"isvc-logger-raw-cipn-predictor-d64bbfccf-j2dsg\" (UID: \"ebde8ce5-b706-4874-be47-578925b6509c\") " pod="e2e-gw0-f92f81a7ef/isvc-logger-raw-cipn-predictor-d64bbfccf-j2dsg"
Oct 30 22:23:12.052560 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:23:12.052439    6133 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/ebde8ce5-b706-4874-be47-578925b6509c-proxy-tls\") pod \"isvc-logger-raw-cipn-predictor-d64bbfccf-j2dsg\" (UID: \"ebde8ce5-b706-4874-be47-578925b6509c\") " pod="e2e-gw0-f92f81a7ef/isvc-logger-raw-cipn-predictor-d64bbfccf-j2dsg"
Oct 30 22:23:12.052560 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:23:12.052472    6133 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/ebde8ce5-b706-4874-be47-578925b6509c-cabundle-cert\") pod \"isvc-logger-raw-cipn-predictor-d64bbfccf-j2dsg\" (UID: \"ebde8ce5-b706-4874-be47-578925b6509c\") " pod="e2e-gw0-f92f81a7ef/isvc-logger-raw-cipn-predictor-d64bbfccf-j2dsg"
Oct 30 22:23:12.052857 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:23:12.052833    6133 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/ebde8ce5-b706-4874-be47-578925b6509c-kserve-provision-location\") pod \"isvc-logger-raw-cipn-predictor-d64bbfccf-j2dsg\" (UID: \"ebde8ce5-b706-4874-be47-578925b6509c\") " pod="e2e-gw0-f92f81a7ef/isvc-logger-raw-cipn-predictor-d64bbfccf-j2dsg"
Oct 30 22:23:12.053127 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:23:12.053106    6133 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/ebde8ce5-b706-4874-be47-578925b6509c-kube-rbac-proxy-sar-config\") pod \"isvc-logger-raw-cipn-predictor-d64bbfccf-j2dsg\" (UID: \"ebde8ce5-b706-4874-be47-578925b6509c\") " pod="e2e-gw0-f92f81a7ef/isvc-logger-raw-cipn-predictor-d64bbfccf-j2dsg"
Oct 30 22:23:12.053204 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:23:12.053163    6133 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/ebde8ce5-b706-4874-be47-578925b6509c-cabundle-cert\") pod \"isvc-logger-raw-cipn-predictor-d64bbfccf-j2dsg\" (UID: \"ebde8ce5-b706-4874-be47-578925b6509c\") " pod="e2e-gw0-f92f81a7ef/isvc-logger-raw-cipn-predictor-d64bbfccf-j2dsg"
Oct 30 22:23:12.054846 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:23:12.054828    6133 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/ebde8ce5-b706-4874-be47-578925b6509c-proxy-tls\") pod \"isvc-logger-raw-cipn-predictor-d64bbfccf-j2dsg\" (UID: \"ebde8ce5-b706-4874-be47-578925b6509c\") " pod="e2e-gw0-f92f81a7ef/isvc-logger-raw-cipn-predictor-d64bbfccf-j2dsg"
Oct 30 22:23:12.073751 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:23:12.073715    6133 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2zpcj\" (UniqueName: \"kubernetes.io/projected/ebde8ce5-b706-4874-be47-578925b6509c-kube-api-access-2zpcj\") pod \"isvc-logger-raw-cipn-predictor-d64bbfccf-j2dsg\" (UID: \"ebde8ce5-b706-4874-be47-578925b6509c\") " pod="e2e-gw0-f92f81a7ef/isvc-logger-raw-cipn-predictor-d64bbfccf-j2dsg"
Oct 30 22:23:12.206260 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:23:12.206159    6133 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-f92f81a7ef/isvc-logger-raw-cipn-predictor-d64bbfccf-j2dsg"
Oct 30 22:23:12.523327 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:23:12.523297    6133 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw0-f92f81a7ef/isvc-logger-raw-cipn-predictor-d64bbfccf-j2dsg"]
Oct 30 22:23:12.523827 ip-10-0-143-246 kubenswrapper[6133]: W1030 22:23:12.523803    6133 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podebde8ce5_b706_4874_be47_578925b6509c.slice/crio-8e1d6a6ed35cd9f123676641039bf5ebf5fdc545f664c71346983cda0f37f3a7 WatchSource:0}: Error finding container 8e1d6a6ed35cd9f123676641039bf5ebf5fdc545f664c71346983cda0f37f3a7: Status 404 returned error can't find the container with id 8e1d6a6ed35cd9f123676641039bf5ebf5fdc545f664c71346983cda0f37f3a7
Oct 30 22:23:12.948780 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:23:12.948682    6133 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-f92f81a7ef/isvc-logger-raw-cipn-predictor-d64bbfccf-j2dsg" event={"ID":"ebde8ce5-b706-4874-be47-578925b6509c","Type":"ContainerStarted","Data":"d64c07ec3ef14daeee68be672afc4b49227b44e0f51c81cc4577ffa60d2e5a2c"}
Oct 30 22:23:12.948780 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:23:12.948749    6133 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-f92f81a7ef/isvc-logger-raw-cipn-predictor-d64bbfccf-j2dsg" event={"ID":"ebde8ce5-b706-4874-be47-578925b6509c","Type":"ContainerStarted","Data":"8e1d6a6ed35cd9f123676641039bf5ebf5fdc545f664c71346983cda0f37f3a7"}
Oct 30 22:23:16.964026 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:23:16.963965    6133 generic.go:356] "Generic (PLEG): container finished" podID="ebde8ce5-b706-4874-be47-578925b6509c" containerID="d64c07ec3ef14daeee68be672afc4b49227b44e0f51c81cc4577ffa60d2e5a2c" exitCode=0
Oct 30 22:23:16.964419 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:23:16.964034    6133 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-f92f81a7ef/isvc-logger-raw-cipn-predictor-d64bbfccf-j2dsg" event={"ID":"ebde8ce5-b706-4874-be47-578925b6509c","Type":"ContainerDied","Data":"d64c07ec3ef14daeee68be672afc4b49227b44e0f51c81cc4577ffa60d2e5a2c"}
Oct 30 22:23:17.969593 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:23:17.969559    6133 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-f92f81a7ef/isvc-logger-raw-cipn-predictor-d64bbfccf-j2dsg" event={"ID":"ebde8ce5-b706-4874-be47-578925b6509c","Type":"ContainerStarted","Data":"49d078d848b160613764e5849c9bb6a7ab8fcc789f4aa84e3f60ad9eee561db5"}
Oct 30 22:23:17.970050 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:23:17.969624    6133 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-f92f81a7ef/isvc-logger-raw-cipn-predictor-d64bbfccf-j2dsg" event={"ID":"ebde8ce5-b706-4874-be47-578925b6509c","Type":"ContainerStarted","Data":"3b6ad6aec692a2a534b0d2015f70955a126ba79b6140dcdbbaaf7ca1c0565334"}
Oct 30 22:23:17.970050 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:23:17.969648    6133 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-f92f81a7ef/isvc-logger-raw-cipn-predictor-d64bbfccf-j2dsg" event={"ID":"ebde8ce5-b706-4874-be47-578925b6509c","Type":"ContainerStarted","Data":"75566b139306528d712c1f4ed8ad92222dc4ea26a47115fd5f9ee4e9f772b3c8"}
Oct 30 22:23:17.970154 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:23:17.970048    6133 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw0-f92f81a7ef/isvc-logger-raw-cipn-predictor-d64bbfccf-j2dsg"
Oct 30 22:23:17.970154 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:23:17.970111    6133 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw0-f92f81a7ef/isvc-logger-raw-cipn-predictor-d64bbfccf-j2dsg"
Oct 30 22:23:17.970154 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:23:17.970144    6133 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw0-f92f81a7ef/isvc-logger-raw-cipn-predictor-d64bbfccf-j2dsg"
Oct 30 22:23:17.971652 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:23:17.971627    6133 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-f92f81a7ef/isvc-logger-raw-cipn-predictor-d64bbfccf-j2dsg" podUID="ebde8ce5-b706-4874-be47-578925b6509c" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.34:8080: connect: connection refused"
Oct 30 22:23:17.972286 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:23:17.972256    6133 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-f92f81a7ef/isvc-logger-raw-cipn-predictor-d64bbfccf-j2dsg" podUID="ebde8ce5-b706-4874-be47-578925b6509c" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 22:23:18.041800 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:23:18.041747    6133 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-gw0-f92f81a7ef/isvc-logger-raw-cipn-predictor-d64bbfccf-j2dsg" podStartSLOduration=3.410777901 podStartE2EDuration="7.041733302s" totalImagesPullingTime="0s" totalInitContainerRuntime="3.630955401s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:23:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:23:18.02887541 +0000 UTC m=+1704.789631300" watchObservedRunningTime="2026-10-30 22:23:18.041733302 +0000 UTC m=+1704.802489191"
Oct 30 22:23:18.974866 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:23:18.974821    6133 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-f92f81a7ef/isvc-logger-raw-cipn-predictor-d64bbfccf-j2dsg" podUID="ebde8ce5-b706-4874-be47-578925b6509c" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.34:8080: connect: connection refused"
Oct 30 22:23:18.975344 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:23:18.975216    6133 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-f92f81a7ef/isvc-logger-raw-cipn-predictor-d64bbfccf-j2dsg" podUID="ebde8ce5-b706-4874-be47-578925b6509c" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 22:23:23.980006 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:23:23.979953    6133 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw0-f92f81a7ef/isvc-logger-raw-cipn-predictor-d64bbfccf-j2dsg"
Oct 30 22:23:23.980821 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:23:23.980776    6133 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-f92f81a7ef/isvc-logger-raw-cipn-predictor-d64bbfccf-j2dsg" podUID="ebde8ce5-b706-4874-be47-578925b6509c" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.34:8080: connect: connection refused"
Oct 30 22:23:23.980951 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:23:23.980832    6133 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-f92f81a7ef/isvc-logger-raw-cipn-predictor-d64bbfccf-j2dsg" podUID="ebde8ce5-b706-4874-be47-578925b6509c" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 22:23:33.980981 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:23:33.980928    6133 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-f92f81a7ef/isvc-logger-raw-cipn-predictor-d64bbfccf-j2dsg" podUID="ebde8ce5-b706-4874-be47-578925b6509c" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.34:8080: connect: connection refused"
Oct 30 22:23:33.981439 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:23:33.981360    6133 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-f92f81a7ef/isvc-logger-raw-cipn-predictor-d64bbfccf-j2dsg" podUID="ebde8ce5-b706-4874-be47-578925b6509c" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 22:23:39.528197 ip-10-0-143-246 kubenswrapper[6133]: E1030 22:23:39.528114    6133 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6jppj" containerName="extractor"
Oct 30 22:23:43.981265 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:23:43.981217    6133 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-f92f81a7ef/isvc-logger-raw-cipn-predictor-d64bbfccf-j2dsg" podUID="ebde8ce5-b706-4874-be47-578925b6509c" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.34:8080: connect: connection refused"
Oct 30 22:23:43.981751 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:23:43.981718    6133 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-f92f81a7ef/isvc-logger-raw-cipn-predictor-d64bbfccf-j2dsg" podUID="ebde8ce5-b706-4874-be47-578925b6509c" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 22:23:53.981407 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:23:53.981364    6133 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-f92f81a7ef/isvc-logger-raw-cipn-predictor-d64bbfccf-j2dsg" podUID="ebde8ce5-b706-4874-be47-578925b6509c" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.34:8080: connect: connection refused"
Oct 30 22:23:53.981890 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:23:53.981869    6133 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-f92f81a7ef/isvc-logger-raw-cipn-predictor-d64bbfccf-j2dsg" podUID="ebde8ce5-b706-4874-be47-578925b6509c" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 22:24:03.980671 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:24:03.980625    6133 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-f92f81a7ef/isvc-logger-raw-cipn-predictor-d64bbfccf-j2dsg" podUID="ebde8ce5-b706-4874-be47-578925b6509c" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.34:8080: connect: connection refused"
Oct 30 22:24:03.981124 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:24:03.981097    6133 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-f92f81a7ef/isvc-logger-raw-cipn-predictor-d64bbfccf-j2dsg" podUID="ebde8ce5-b706-4874-be47-578925b6509c" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 22:24:09.527214 ip-10-0-143-246 kubenswrapper[6133]: E1030 22:24:09.527172    6133 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6jppj" containerName="extractor"
Oct 30 22:24:13.980684 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:24:13.980646    6133 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-f92f81a7ef/isvc-logger-raw-cipn-predictor-d64bbfccf-j2dsg" podUID="ebde8ce5-b706-4874-be47-578925b6509c" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.34:8080: connect: connection refused"
Oct 30 22:24:13.981284 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:24:13.981141    6133 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-f92f81a7ef/isvc-logger-raw-cipn-predictor-d64bbfccf-j2dsg" podUID="ebde8ce5-b706-4874-be47-578925b6509c" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 22:24:23.980480 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:24:23.980439    6133 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-f92f81a7ef/isvc-logger-raw-cipn-predictor-d64bbfccf-j2dsg" podUID="ebde8ce5-b706-4874-be47-578925b6509c" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.34:8080: connect: connection refused"
Oct 30 22:24:23.982772 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:24:23.980941    6133 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-f92f81a7ef/isvc-logger-raw-cipn-predictor-d64bbfccf-j2dsg" podUID="ebde8ce5-b706-4874-be47-578925b6509c" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 22:24:33.981062 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:24:33.981013    6133 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-f92f81a7ef/isvc-logger-raw-cipn-predictor-d64bbfccf-j2dsg" podUID="ebde8ce5-b706-4874-be47-578925b6509c" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.34:8080: connect: connection refused"
Oct 30 22:24:33.981548 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:24:33.981459    6133 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-f92f81a7ef/isvc-logger-raw-cipn-predictor-d64bbfccf-j2dsg" podUID="ebde8ce5-b706-4874-be47-578925b6509c" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 22:24:39.527582 ip-10-0-143-246 kubenswrapper[6133]: E1030 22:24:39.527545    6133 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6jppj" containerName="extractor"
Oct 30 22:24:43.354865 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:24:43.354839    6133 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw0-f92f81a7ef/isvc-logger-raw-cipn-predictor-d64bbfccf-j2dsg"
Oct 30 22:24:43.355261 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:24:43.354913    6133 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw0-f92f81a7ef/isvc-logger-raw-cipn-predictor-d64bbfccf-j2dsg"
Oct 30 22:24:57.242168 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:24:57.242139    6133 log.go:25] "Finished parsing log file" path="/var/log/pods/e2e-gw0-f92f81a7ef_message-dumper-raw-cipn-predictor-57464d98d7-scpm6_864efcb3-72df-4a51-84ec-fa24de8ab96b/kserve-container/0.log"
Oct 30 22:24:57.881342 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:24:57.881312    6133 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw0-f92f81a7ef/isvc-logger-raw-cipn-predictor-d64bbfccf-j2dsg"]
Oct 30 22:24:57.881624 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:24:57.881589    6133 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw0-f92f81a7ef/isvc-logger-raw-cipn-predictor-d64bbfccf-j2dsg" podUID="ebde8ce5-b706-4874-be47-578925b6509c" containerName="kserve-container" containerID="cri-o://75566b139306528d712c1f4ed8ad92222dc4ea26a47115fd5f9ee4e9f772b3c8" gracePeriod=30
Oct 30 22:24:57.881786 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:24:57.881644    6133 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw0-f92f81a7ef/isvc-logger-raw-cipn-predictor-d64bbfccf-j2dsg" podUID="ebde8ce5-b706-4874-be47-578925b6509c" containerName="agent" containerID="cri-o://49d078d848b160613764e5849c9bb6a7ab8fcc789f4aa84e3f60ad9eee561db5" gracePeriod=30
Oct 30 22:24:57.881786 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:24:57.881656    6133 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw0-f92f81a7ef/isvc-logger-raw-cipn-predictor-d64bbfccf-j2dsg" podUID="ebde8ce5-b706-4874-be47-578925b6509c" containerName="kube-rbac-proxy" containerID="cri-o://3b6ad6aec692a2a534b0d2015f70955a126ba79b6140dcdbbaaf7ca1c0565334" gracePeriod=30
Oct 30 22:24:57.988586 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:24:57.988551    6133 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw0-f92f81a7ef/message-dumper-raw-cipn-predictor-57464d98d7-scpm6"]
Oct 30 22:24:57.988826 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:24:57.988799    6133 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw0-f92f81a7ef/message-dumper-raw-cipn-predictor-57464d98d7-scpm6" podUID="864efcb3-72df-4a51-84ec-fa24de8ab96b" containerName="kserve-container" containerID="cri-o://883005143fd3c459fc961107352e421f7a5d163ed9c8dd7be677f1a9f1920923" gracePeriod=30
Oct 30 22:24:57.988922 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:24:57.988892    6133 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw0-f92f81a7ef/message-dumper-raw-cipn-predictor-57464d98d7-scpm6" podUID="864efcb3-72df-4a51-84ec-fa24de8ab96b" containerName="kube-rbac-proxy" containerID="cri-o://e4eac2bfa245c06bf7aee006330d161eb6274803e7595f4eb4fd1d29d304af7a" gracePeriod=30
Oct 30 22:24:58.230297 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:24:58.230268    6133 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-f92f81a7ef/message-dumper-raw-cipn-predictor-57464d98d7-scpm6"
Oct 30 22:24:58.348266 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:24:58.348232    6133 generic.go:356] "Generic (PLEG): container finished" podID="864efcb3-72df-4a51-84ec-fa24de8ab96b" containerID="e4eac2bfa245c06bf7aee006330d161eb6274803e7595f4eb4fd1d29d304af7a" exitCode=0
Oct 30 22:24:58.348266 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:24:58.348257    6133 generic.go:356] "Generic (PLEG): container finished" podID="864efcb3-72df-4a51-84ec-fa24de8ab96b" containerID="883005143fd3c459fc961107352e421f7a5d163ed9c8dd7be677f1a9f1920923" exitCode=2
Oct 30 22:24:58.348748 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:24:58.348283    6133 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-f92f81a7ef/message-dumper-raw-cipn-predictor-57464d98d7-scpm6"
Oct 30 22:24:58.348748 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:24:58.348319    6133 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-f92f81a7ef/message-dumper-raw-cipn-predictor-57464d98d7-scpm6" event={"ID":"864efcb3-72df-4a51-84ec-fa24de8ab96b","Type":"ContainerDied","Data":"e4eac2bfa245c06bf7aee006330d161eb6274803e7595f4eb4fd1d29d304af7a"}
Oct 30 22:24:58.348748 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:24:58.348385    6133 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-f92f81a7ef/message-dumper-raw-cipn-predictor-57464d98d7-scpm6" event={"ID":"864efcb3-72df-4a51-84ec-fa24de8ab96b","Type":"ContainerDied","Data":"883005143fd3c459fc961107352e421f7a5d163ed9c8dd7be677f1a9f1920923"}
Oct 30 22:24:58.348748 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:24:58.348415    6133 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-f92f81a7ef/message-dumper-raw-cipn-predictor-57464d98d7-scpm6" event={"ID":"864efcb3-72df-4a51-84ec-fa24de8ab96b","Type":"ContainerDied","Data":"f8cec0d203df39aec11d3de0179b79fb7b7442d78acdca545616630373a57cdf"}
Oct 30 22:24:58.348748 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:24:58.348445    6133 scope.go:122] "RemoveContainer" containerID="e4eac2bfa245c06bf7aee006330d161eb6274803e7595f4eb4fd1d29d304af7a"
Oct 30 22:24:58.350880 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:24:58.350861    6133 generic.go:356] "Generic (PLEG): container finished" podID="ebde8ce5-b706-4874-be47-578925b6509c" containerID="3b6ad6aec692a2a534b0d2015f70955a126ba79b6140dcdbbaaf7ca1c0565334" exitCode=0
Oct 30 22:24:58.351018 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:24:58.350934    6133 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-f92f81a7ef/isvc-logger-raw-cipn-predictor-d64bbfccf-j2dsg" event={"ID":"ebde8ce5-b706-4874-be47-578925b6509c","Type":"ContainerDied","Data":"3b6ad6aec692a2a534b0d2015f70955a126ba79b6140dcdbbaaf7ca1c0565334"}
Oct 30 22:24:58.356804 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:24:58.356785    6133 scope.go:122] "RemoveContainer" containerID="883005143fd3c459fc961107352e421f7a5d163ed9c8dd7be677f1a9f1920923"
Oct 30 22:24:58.364603 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:24:58.364585    6133 scope.go:122] "RemoveContainer" containerID="e4eac2bfa245c06bf7aee006330d161eb6274803e7595f4eb4fd1d29d304af7a"
Oct 30 22:24:58.364860 ip-10-0-143-246 kubenswrapper[6133]: E1030 22:24:58.364843    6133 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e4eac2bfa245c06bf7aee006330d161eb6274803e7595f4eb4fd1d29d304af7a\": container with ID starting with e4eac2bfa245c06bf7aee006330d161eb6274803e7595f4eb4fd1d29d304af7a not found: ID does not exist" containerID="e4eac2bfa245c06bf7aee006330d161eb6274803e7595f4eb4fd1d29d304af7a"
Oct 30 22:24:58.364908 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:24:58.364868    6133 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e4eac2bfa245c06bf7aee006330d161eb6274803e7595f4eb4fd1d29d304af7a"} err="failed to get container status \"e4eac2bfa245c06bf7aee006330d161eb6274803e7595f4eb4fd1d29d304af7a\": rpc error: code = NotFound desc = could not find container \"e4eac2bfa245c06bf7aee006330d161eb6274803e7595f4eb4fd1d29d304af7a\": container with ID starting with e4eac2bfa245c06bf7aee006330d161eb6274803e7595f4eb4fd1d29d304af7a not found: ID does not exist"
Oct 30 22:24:58.364908 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:24:58.364883    6133 scope.go:122] "RemoveContainer" containerID="883005143fd3c459fc961107352e421f7a5d163ed9c8dd7be677f1a9f1920923"
Oct 30 22:24:58.365129 ip-10-0-143-246 kubenswrapper[6133]: E1030 22:24:58.365115    6133 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"883005143fd3c459fc961107352e421f7a5d163ed9c8dd7be677f1a9f1920923\": container with ID starting with 883005143fd3c459fc961107352e421f7a5d163ed9c8dd7be677f1a9f1920923 not found: ID does not exist" containerID="883005143fd3c459fc961107352e421f7a5d163ed9c8dd7be677f1a9f1920923"
Oct 30 22:24:58.365182 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:24:58.365132    6133 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"883005143fd3c459fc961107352e421f7a5d163ed9c8dd7be677f1a9f1920923"} err="failed to get container status \"883005143fd3c459fc961107352e421f7a5d163ed9c8dd7be677f1a9f1920923\": rpc error: code = NotFound desc = could not find container \"883005143fd3c459fc961107352e421f7a5d163ed9c8dd7be677f1a9f1920923\": container with ID starting with 883005143fd3c459fc961107352e421f7a5d163ed9c8dd7be677f1a9f1920923 not found: ID does not exist"
Oct 30 22:24:58.365182 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:24:58.365144    6133 scope.go:122] "RemoveContainer" containerID="e4eac2bfa245c06bf7aee006330d161eb6274803e7595f4eb4fd1d29d304af7a"
Oct 30 22:24:58.365357 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:24:58.365335    6133 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e4eac2bfa245c06bf7aee006330d161eb6274803e7595f4eb4fd1d29d304af7a"} err="failed to get container status \"e4eac2bfa245c06bf7aee006330d161eb6274803e7595f4eb4fd1d29d304af7a\": rpc error: code = NotFound desc = could not find container \"e4eac2bfa245c06bf7aee006330d161eb6274803e7595f4eb4fd1d29d304af7a\": container with ID starting with e4eac2bfa245c06bf7aee006330d161eb6274803e7595f4eb4fd1d29d304af7a not found: ID does not exist"
Oct 30 22:24:58.365429 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:24:58.365359    6133 scope.go:122] "RemoveContainer" containerID="883005143fd3c459fc961107352e421f7a5d163ed9c8dd7be677f1a9f1920923"
Oct 30 22:24:58.365576 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:24:58.365559    6133 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"883005143fd3c459fc961107352e421f7a5d163ed9c8dd7be677f1a9f1920923"} err="failed to get container status \"883005143fd3c459fc961107352e421f7a5d163ed9c8dd7be677f1a9f1920923\": rpc error: code = NotFound desc = could not find container \"883005143fd3c459fc961107352e421f7a5d163ed9c8dd7be677f1a9f1920923\": container with ID starting with 883005143fd3c459fc961107352e421f7a5d163ed9c8dd7be677f1a9f1920923 not found: ID does not exist"
Oct 30 22:24:58.390059 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:24:58.390028    6133 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/864efcb3-72df-4a51-84ec-fa24de8ab96b-kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/864efcb3-72df-4a51-84ec-fa24de8ab96b-kube-rbac-proxy-sar-config\") pod \"864efcb3-72df-4a51-84ec-fa24de8ab96b\" (UID: \"864efcb3-72df-4a51-84ec-fa24de8ab96b\") "
Oct 30 22:24:58.390215 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:24:58.390081    6133 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/864efcb3-72df-4a51-84ec-fa24de8ab96b-kube-api-access-qxqwt\" (UniqueName: \"kubernetes.io/projected/864efcb3-72df-4a51-84ec-fa24de8ab96b-kube-api-access-qxqwt\") pod \"864efcb3-72df-4a51-84ec-fa24de8ab96b\" (UID: \"864efcb3-72df-4a51-84ec-fa24de8ab96b\") "
Oct 30 22:24:58.390215 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:24:58.390132    6133 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/864efcb3-72df-4a51-84ec-fa24de8ab96b-proxy-tls\" (UniqueName: \"kubernetes.io/secret/864efcb3-72df-4a51-84ec-fa24de8ab96b-proxy-tls\") pod \"864efcb3-72df-4a51-84ec-fa24de8ab96b\" (UID: \"864efcb3-72df-4a51-84ec-fa24de8ab96b\") "
Oct 30 22:24:58.390464 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:24:58.390441    6133 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/864efcb3-72df-4a51-84ec-fa24de8ab96b-kube-rbac-proxy-sar-config" pod "864efcb3-72df-4a51-84ec-fa24de8ab96b" (UID: "864efcb3-72df-4a51-84ec-fa24de8ab96b"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:24:58.392340 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:24:58.392318    6133 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/864efcb3-72df-4a51-84ec-fa24de8ab96b-proxy-tls" pod "864efcb3-72df-4a51-84ec-fa24de8ab96b" (UID: "864efcb3-72df-4a51-84ec-fa24de8ab96b"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:24:58.392340 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:24:58.392331    6133 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/864efcb3-72df-4a51-84ec-fa24de8ab96b-kube-api-access-qxqwt" pod "864efcb3-72df-4a51-84ec-fa24de8ab96b" (UID: "864efcb3-72df-4a51-84ec-fa24de8ab96b"). InnerVolumeSpecName "kube-api-access-qxqwt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:24:58.491453 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:24:58.491372    6133 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qxqwt\" (UniqueName: \"kubernetes.io/projected/864efcb3-72df-4a51-84ec-fa24de8ab96b-kube-api-access-qxqwt\") on node \"ip-10-0-143-246.ec2.internal\" DevicePath \"\""
Oct 30 22:24:58.491453 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:24:58.491400    6133 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/864efcb3-72df-4a51-84ec-fa24de8ab96b-proxy-tls\") on node \"ip-10-0-143-246.ec2.internal\" DevicePath \"\""
Oct 30 22:24:58.491453 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:24:58.491410    6133 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/864efcb3-72df-4a51-84ec-fa24de8ab96b-kube-rbac-proxy-sar-config\") on node \"ip-10-0-143-246.ec2.internal\" DevicePath \"\""
Oct 30 22:24:58.696051 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:24:58.696017    6133 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw0-f92f81a7ef/message-dumper-raw-cipn-predictor-57464d98d7-scpm6"]
Oct 30 22:24:58.698525 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:24:58.698502    6133 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-gw0-f92f81a7ef/message-dumper-raw-cipn-predictor-57464d98d7-scpm6"]
Oct 30 22:24:58.975914 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:24:58.975872    6133 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-f92f81a7ef/isvc-logger-raw-cipn-predictor-d64bbfccf-j2dsg" podUID="ebde8ce5-b706-4874-be47-578925b6509c" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.34:8643/healthz\": dial tcp 10.134.0.34:8643: connect: connection refused"
Oct 30 22:24:59.354803 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:24:59.354764    6133 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="864efcb3-72df-4a51-84ec-fa24de8ab96b" path="/var/lib/kubelet/pods/864efcb3-72df-4a51-84ec-fa24de8ab96b/volumes"
Oct 30 22:25:03.353670 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:25:03.353632    6133 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-f92f81a7ef/isvc-logger-raw-cipn-predictor-d64bbfccf-j2dsg" podUID="ebde8ce5-b706-4874-be47-578925b6509c" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.34:8080: connect: connection refused"
Oct 30 22:25:03.354158 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:25:03.354123    6133 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-f92f81a7ef/isvc-logger-raw-cipn-predictor-d64bbfccf-j2dsg" podUID="ebde8ce5-b706-4874-be47-578925b6509c" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 22:25:03.372207 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:25:03.372171    6133 generic.go:356] "Generic (PLEG): container finished" podID="ebde8ce5-b706-4874-be47-578925b6509c" containerID="75566b139306528d712c1f4ed8ad92222dc4ea26a47115fd5f9ee4e9f772b3c8" exitCode=0
Oct 30 22:25:03.372390 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:25:03.372244    6133 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-f92f81a7ef/isvc-logger-raw-cipn-predictor-d64bbfccf-j2dsg" event={"ID":"ebde8ce5-b706-4874-be47-578925b6509c","Type":"ContainerDied","Data":"75566b139306528d712c1f4ed8ad92222dc4ea26a47115fd5f9ee4e9f772b3c8"}
Oct 30 22:25:03.975595 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:25:03.975556    6133 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-f92f81a7ef/isvc-logger-raw-cipn-predictor-d64bbfccf-j2dsg" podUID="ebde8ce5-b706-4874-be47-578925b6509c" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.34:8643/healthz\": dial tcp 10.134.0.34:8643: connect: connection refused"
Oct 30 22:25:08.975235 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:25:08.975183    6133 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-f92f81a7ef/isvc-logger-raw-cipn-predictor-d64bbfccf-j2dsg" podUID="ebde8ce5-b706-4874-be47-578925b6509c" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.34:8643/healthz\": dial tcp 10.134.0.34:8643: connect: connection refused"
Oct 30 22:25:08.975689 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:25:08.975350    6133 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw0-f92f81a7ef/isvc-logger-raw-cipn-predictor-d64bbfccf-j2dsg"
Oct 30 22:25:09.527255 ip-10-0-143-246 kubenswrapper[6133]: E1030 22:25:09.527221    6133 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6jppj" containerName="extractor"
Oct 30 22:25:13.352680 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:25:13.352631    6133 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-f92f81a7ef/isvc-logger-raw-cipn-predictor-d64bbfccf-j2dsg" podUID="ebde8ce5-b706-4874-be47-578925b6509c" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.34:8080: connect: connection refused"
Oct 30 22:25:13.353155 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:25:13.352947    6133 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-f92f81a7ef/isvc-logger-raw-cipn-predictor-d64bbfccf-j2dsg" podUID="ebde8ce5-b706-4874-be47-578925b6509c" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 22:25:13.975943 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:25:13.975890    6133 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-f92f81a7ef/isvc-logger-raw-cipn-predictor-d64bbfccf-j2dsg" podUID="ebde8ce5-b706-4874-be47-578925b6509c" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.34:8643/healthz\": dial tcp 10.134.0.34:8643: connect: connection refused"
Oct 30 22:25:18.975029 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:25:18.974956    6133 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-f92f81a7ef/isvc-logger-raw-cipn-predictor-d64bbfccf-j2dsg" podUID="ebde8ce5-b706-4874-be47-578925b6509c" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.34:8643/healthz\": dial tcp 10.134.0.34:8643: connect: connection refused"
Oct 30 22:25:23.352664 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:25:23.352611    6133 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-f92f81a7ef/isvc-logger-raw-cipn-predictor-d64bbfccf-j2dsg" podUID="ebde8ce5-b706-4874-be47-578925b6509c" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.34:8080: connect: connection refused"
Oct 30 22:25:23.357520 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:25:23.357484    6133 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-f92f81a7ef/isvc-logger-raw-cipn-predictor-d64bbfccf-j2dsg" podUID="ebde8ce5-b706-4874-be47-578925b6509c" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 22:25:23.360152 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:25:23.360130    6133 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw0-f92f81a7ef/isvc-logger-raw-cipn-predictor-d64bbfccf-j2dsg"
Oct 30 22:25:23.360261 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:25:23.360219    6133 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw0-f92f81a7ef/isvc-logger-raw-cipn-predictor-d64bbfccf-j2dsg"
Oct 30 22:25:23.975392 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:25:23.975349    6133 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-f92f81a7ef/isvc-logger-raw-cipn-predictor-d64bbfccf-j2dsg" podUID="ebde8ce5-b706-4874-be47-578925b6509c" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.34:8643/healthz\": dial tcp 10.134.0.34:8643: connect: connection refused"
Oct 30 22:25:28.463279 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:25:28.463248    6133 generic.go:356] "Generic (PLEG): container finished" podID="ebde8ce5-b706-4874-be47-578925b6509c" containerID="49d078d848b160613764e5849c9bb6a7ab8fcc789f4aa84e3f60ad9eee561db5" exitCode=0
Oct 30 22:25:28.463689 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:25:28.463323    6133 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-f92f81a7ef/isvc-logger-raw-cipn-predictor-d64bbfccf-j2dsg" event={"ID":"ebde8ce5-b706-4874-be47-578925b6509c","Type":"ContainerDied","Data":"49d078d848b160613764e5849c9bb6a7ab8fcc789f4aa84e3f60ad9eee561db5"}
Oct 30 22:25:28.528890 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:25:28.528866    6133 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-f92f81a7ef/isvc-logger-raw-cipn-predictor-d64bbfccf-j2dsg"
Oct 30 22:25:28.629776 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:25:28.629745    6133 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ebde8ce5-b706-4874-be47-578925b6509c-cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/ebde8ce5-b706-4874-be47-578925b6509c-cabundle-cert\") pod \"ebde8ce5-b706-4874-be47-578925b6509c\" (UID: \"ebde8ce5-b706-4874-be47-578925b6509c\") "
Oct 30 22:25:28.629776 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:25:28.629780    6133 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ebde8ce5-b706-4874-be47-578925b6509c-kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/ebde8ce5-b706-4874-be47-578925b6509c-kube-rbac-proxy-sar-config\") pod \"ebde8ce5-b706-4874-be47-578925b6509c\" (UID: \"ebde8ce5-b706-4874-be47-578925b6509c\") "
Oct 30 22:25:28.630025 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:25:28.629805    6133 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ebde8ce5-b706-4874-be47-578925b6509c-kube-api-access-2zpcj\" (UniqueName: \"kubernetes.io/projected/ebde8ce5-b706-4874-be47-578925b6509c-kube-api-access-2zpcj\") pod \"ebde8ce5-b706-4874-be47-578925b6509c\" (UID: \"ebde8ce5-b706-4874-be47-578925b6509c\") "
Oct 30 22:25:28.630025 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:25:28.629837    6133 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/ebde8ce5-b706-4874-be47-578925b6509c-proxy-tls\" (UniqueName: \"kubernetes.io/secret/ebde8ce5-b706-4874-be47-578925b6509c-proxy-tls\") pod \"ebde8ce5-b706-4874-be47-578925b6509c\" (UID: \"ebde8ce5-b706-4874-be47-578925b6509c\") "
Oct 30 22:25:28.630025 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:25:28.629864    6133 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ebde8ce5-b706-4874-be47-578925b6509c-kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/ebde8ce5-b706-4874-be47-578925b6509c-kserve-provision-location\") pod \"ebde8ce5-b706-4874-be47-578925b6509c\" (UID: \"ebde8ce5-b706-4874-be47-578925b6509c\") "
Oct 30 22:25:28.630280 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:25:28.630246    6133 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ebde8ce5-b706-4874-be47-578925b6509c-kserve-provision-location" pod "ebde8ce5-b706-4874-be47-578925b6509c" (UID: "ebde8ce5-b706-4874-be47-578925b6509c"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:25:28.630280 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:25:28.630260    6133 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ebde8ce5-b706-4874-be47-578925b6509c-cabundle-cert" pod "ebde8ce5-b706-4874-be47-578925b6509c" (UID: "ebde8ce5-b706-4874-be47-578925b6509c"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:25:28.630450 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:25:28.630292    6133 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ebde8ce5-b706-4874-be47-578925b6509c-kube-rbac-proxy-sar-config" pod "ebde8ce5-b706-4874-be47-578925b6509c" (UID: "ebde8ce5-b706-4874-be47-578925b6509c"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:25:28.632137 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:25:28.632107    6133 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ebde8ce5-b706-4874-be47-578925b6509c-kube-api-access-2zpcj" pod "ebde8ce5-b706-4874-be47-578925b6509c" (UID: "ebde8ce5-b706-4874-be47-578925b6509c"). InnerVolumeSpecName "kube-api-access-2zpcj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:25:28.632137 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:25:28.632129    6133 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ebde8ce5-b706-4874-be47-578925b6509c-proxy-tls" pod "ebde8ce5-b706-4874-be47-578925b6509c" (UID: "ebde8ce5-b706-4874-be47-578925b6509c"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:25:28.730634 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:25:28.730539    6133 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/ebde8ce5-b706-4874-be47-578925b6509c-cabundle-cert\") on node \"ip-10-0-143-246.ec2.internal\" DevicePath \"\""
Oct 30 22:25:28.730634 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:25:28.730570    6133 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/ebde8ce5-b706-4874-be47-578925b6509c-kube-rbac-proxy-sar-config\") on node \"ip-10-0-143-246.ec2.internal\" DevicePath \"\""
Oct 30 22:25:28.730634 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:25:28.730580    6133 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2zpcj\" (UniqueName: \"kubernetes.io/projected/ebde8ce5-b706-4874-be47-578925b6509c-kube-api-access-2zpcj\") on node \"ip-10-0-143-246.ec2.internal\" DevicePath \"\""
Oct 30 22:25:28.730634 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:25:28.730590    6133 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/ebde8ce5-b706-4874-be47-578925b6509c-proxy-tls\") on node \"ip-10-0-143-246.ec2.internal\" DevicePath \"\""
Oct 30 22:25:28.730634 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:25:28.730599    6133 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/ebde8ce5-b706-4874-be47-578925b6509c-kserve-provision-location\") on node \"ip-10-0-143-246.ec2.internal\" DevicePath \"\""
Oct 30 22:25:29.469036 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:25:29.469002    6133 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-f92f81a7ef/isvc-logger-raw-cipn-predictor-d64bbfccf-j2dsg" event={"ID":"ebde8ce5-b706-4874-be47-578925b6509c","Type":"ContainerDied","Data":"8e1d6a6ed35cd9f123676641039bf5ebf5fdc545f664c71346983cda0f37f3a7"}
Oct 30 22:25:29.469442 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:25:29.469078    6133 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-f92f81a7ef/isvc-logger-raw-cipn-predictor-d64bbfccf-j2dsg"
Oct 30 22:25:29.469442 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:25:29.469100    6133 scope.go:122] "RemoveContainer" containerID="49d078d848b160613764e5849c9bb6a7ab8fcc789f4aa84e3f60ad9eee561db5"
Oct 30 22:25:29.476946 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:25:29.476923    6133 scope.go:122] "RemoveContainer" containerID="3b6ad6aec692a2a534b0d2015f70955a126ba79b6140dcdbbaaf7ca1c0565334"
Oct 30 22:25:29.484335 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:25:29.484317    6133 scope.go:122] "RemoveContainer" containerID="75566b139306528d712c1f4ed8ad92222dc4ea26a47115fd5f9ee4e9f772b3c8"
Oct 30 22:25:29.491377 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:25:29.491357    6133 scope.go:122] "RemoveContainer" containerID="d64c07ec3ef14daeee68be672afc4b49227b44e0f51c81cc4577ffa60d2e5a2c"
Oct 30 22:25:29.539623 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:25:29.539590    6133 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw0-f92f81a7ef/isvc-logger-raw-cipn-predictor-d64bbfccf-j2dsg"]
Oct 30 22:25:29.552037 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:25:29.552008    6133 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-gw0-f92f81a7ef/isvc-logger-raw-cipn-predictor-d64bbfccf-j2dsg"]
Oct 30 22:25:31.355836 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:25:31.355794    6133 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ebde8ce5-b706-4874-be47-578925b6509c" path="/var/lib/kubelet/pods/ebde8ce5-b706-4874-be47-578925b6509c/volumes"
Oct 30 22:25:32.355940 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:25:32.355911    6133 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-must-gather-67fvs/must-gather-cr4s2"]
Oct 30 22:25:32.356336 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:25:32.356218    6133 cpu_manager.go:414] "containerMap: removing container" podUID="ebde8ce5-b706-4874-be47-578925b6509c" containerName="kube-rbac-proxy"
Oct 30 22:25:32.356336 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:25:32.356230    6133 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ebde8ce5-b706-4874-be47-578925b6509c" containerName="kube-rbac-proxy"
Oct 30 22:25:32.356336 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:25:32.356243    6133 cpu_manager.go:414] "containerMap: removing container" podUID="ebde8ce5-b706-4874-be47-578925b6509c" containerName="kserve-container"
Oct 30 22:25:32.356336 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:25:32.356249    6133 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ebde8ce5-b706-4874-be47-578925b6509c" containerName="kserve-container"
Oct 30 22:25:32.356336 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:25:32.356255    6133 cpu_manager.go:414] "containerMap: removing container" podUID="ebde8ce5-b706-4874-be47-578925b6509c" containerName="agent"
Oct 30 22:25:32.356336 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:25:32.356261    6133 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ebde8ce5-b706-4874-be47-578925b6509c" containerName="agent"
Oct 30 22:25:32.356336 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:25:32.356268    6133 cpu_manager.go:414] "containerMap: removing container" podUID="ebde8ce5-b706-4874-be47-578925b6509c" containerName="storage-initializer"
Oct 30 22:25:32.356336 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:25:32.356272    6133 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ebde8ce5-b706-4874-be47-578925b6509c" containerName="storage-initializer"
Oct 30 22:25:32.356336 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:25:32.356289    6133 cpu_manager.go:414] "containerMap: removing container" podUID="864efcb3-72df-4a51-84ec-fa24de8ab96b" containerName="kube-rbac-proxy"
Oct 30 22:25:32.356336 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:25:32.356294    6133 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="864efcb3-72df-4a51-84ec-fa24de8ab96b" containerName="kube-rbac-proxy"
Oct 30 22:25:32.356336 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:25:32.356303    6133 cpu_manager.go:414] "containerMap: removing container" podUID="864efcb3-72df-4a51-84ec-fa24de8ab96b" containerName="kserve-container"
Oct 30 22:25:32.356336 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:25:32.356308    6133 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="864efcb3-72df-4a51-84ec-fa24de8ab96b" containerName="kserve-container"
Oct 30 22:25:32.356732 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:25:32.356362    6133 memory_manager.go:365] "RemoveStaleState removing state" podUID="ebde8ce5-b706-4874-be47-578925b6509c" containerName="kube-rbac-proxy"
Oct 30 22:25:32.356732 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:25:32.356368    6133 memory_manager.go:365] "RemoveStaleState removing state" podUID="ebde8ce5-b706-4874-be47-578925b6509c" containerName="kserve-container"
Oct 30 22:25:32.356732 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:25:32.356374    6133 memory_manager.go:365] "RemoveStaleState removing state" podUID="ebde8ce5-b706-4874-be47-578925b6509c" containerName="agent"
Oct 30 22:25:32.356732 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:25:32.356382    6133 memory_manager.go:365] "RemoveStaleState removing state" podUID="864efcb3-72df-4a51-84ec-fa24de8ab96b" containerName="kube-rbac-proxy"
Oct 30 22:25:32.356732 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:25:32.356387    6133 memory_manager.go:365] "RemoveStaleState removing state" podUID="864efcb3-72df-4a51-84ec-fa24de8ab96b" containerName="kserve-container"
Oct 30 22:25:32.361161 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:25:32.361140    6133 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-67fvs/must-gather-cr4s2"
Oct 30 22:25:32.376109 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:25:32.376078    6133 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-67fvs/must-gather-cr4s2"]
Oct 30 22:25:32.392137 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:25:32.392109    6133 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-67fvs\"/\"openshift-service-ca.crt\""
Oct 30 22:25:32.392137 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:25:32.392115    6133 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-67fvs\"/\"default-dockercfg-mcbqf\""
Oct 30 22:25:32.392137 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:25:32.392115    6133 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-67fvs\"/\"kube-root-ca.crt\""
Oct 30 22:25:32.459585 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:25:32.459540    6133 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/10a20102-8426-4a13-a2eb-076c5ae75e1e-must-gather-output\") pod \"must-gather-cr4s2\" (UID: \"10a20102-8426-4a13-a2eb-076c5ae75e1e\") " pod="openshift-must-gather-67fvs/must-gather-cr4s2"
Oct 30 22:25:32.459585 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:25:32.459587    6133 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bpwt2\" (UniqueName: \"kubernetes.io/projected/10a20102-8426-4a13-a2eb-076c5ae75e1e-kube-api-access-bpwt2\") pod \"must-gather-cr4s2\" (UID: \"10a20102-8426-4a13-a2eb-076c5ae75e1e\") " pod="openshift-must-gather-67fvs/must-gather-cr4s2"
Oct 30 22:25:32.560557 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:25:32.560516    6133 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/10a20102-8426-4a13-a2eb-076c5ae75e1e-must-gather-output\") pod \"must-gather-cr4s2\" (UID: \"10a20102-8426-4a13-a2eb-076c5ae75e1e\") " pod="openshift-must-gather-67fvs/must-gather-cr4s2"
Oct 30 22:25:32.560557 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:25:32.560553    6133 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bpwt2\" (UniqueName: \"kubernetes.io/projected/10a20102-8426-4a13-a2eb-076c5ae75e1e-kube-api-access-bpwt2\") pod \"must-gather-cr4s2\" (UID: \"10a20102-8426-4a13-a2eb-076c5ae75e1e\") " pod="openshift-must-gather-67fvs/must-gather-cr4s2"
Oct 30 22:25:32.560887 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:25:32.560863    6133 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/10a20102-8426-4a13-a2eb-076c5ae75e1e-must-gather-output\") pod \"must-gather-cr4s2\" (UID: \"10a20102-8426-4a13-a2eb-076c5ae75e1e\") " pod="openshift-must-gather-67fvs/must-gather-cr4s2"
Oct 30 22:25:32.582897 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:25:32.582869    6133 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bpwt2\" (UniqueName: \"kubernetes.io/projected/10a20102-8426-4a13-a2eb-076c5ae75e1e-kube-api-access-bpwt2\") pod \"must-gather-cr4s2\" (UID: \"10a20102-8426-4a13-a2eb-076c5ae75e1e\") " pod="openshift-must-gather-67fvs/must-gather-cr4s2"
Oct 30 22:25:32.688753 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:25:32.688657    6133 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-67fvs/must-gather-cr4s2"
Oct 30 22:25:32.979862 ip-10-0-143-246 kubenswrapper[6133]: W1030 22:25:32.979829    6133 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod10a20102_8426_4a13_a2eb_076c5ae75e1e.slice/crio-19d010e34db38dddbd3974ad732e2a21a52de54e0426676fa99f59c7af98c662 WatchSource:0}: Error finding container 19d010e34db38dddbd3974ad732e2a21a52de54e0426676fa99f59c7af98c662: Status 404 returned error can't find the container with id 19d010e34db38dddbd3974ad732e2a21a52de54e0426676fa99f59c7af98c662
Oct 30 22:25:32.981170 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:25:32.981146    6133 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-67fvs/must-gather-cr4s2"]
Oct 30 22:25:32.981709 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:25:32.981690    6133 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 22:25:33.485311 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:25:33.485274    6133 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-67fvs/must-gather-cr4s2" event={"ID":"10a20102-8426-4a13-a2eb-076c5ae75e1e","Type":"ContainerStarted","Data":"19d010e34db38dddbd3974ad732e2a21a52de54e0426676fa99f59c7af98c662"}
Oct 30 22:25:39.514649 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:25:39.514609    6133 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-67fvs/must-gather-cr4s2" event={"ID":"10a20102-8426-4a13-a2eb-076c5ae75e1e","Type":"ContainerStarted","Data":"ad35f24f3bb95445ff0b48dc1447526640714e17e31930c1712bee2f72c3ba8b"}
Oct 30 22:25:39.515299 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:25:39.514664    6133 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-67fvs/must-gather-cr4s2" event={"ID":"10a20102-8426-4a13-a2eb-076c5ae75e1e","Type":"ContainerStarted","Data":"270102a10128925c01cfea7a72c10abcc5b59f8ddb3b46716ca123d41d25d735"}
Oct 30 22:25:39.535525 ip-10-0-143-246 kubenswrapper[6133]: E1030 22:25:39.535487    6133 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6jppj" containerName="extractor"
Oct 30 22:25:39.556537 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:25:39.556476    6133 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-must-gather-67fvs/must-gather-cr4s2" podStartSLOduration=1.335527103 podStartE2EDuration="7.556461404s" totalImagesPullingTime="6.220934301s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:25:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:25:39.555741068 +0000 UTC m=+1846.316496957" watchObservedRunningTime="2026-10-30 22:25:39.556461404 +0000 UTC m=+1846.317217292"
Oct 30 22:26:08.625878 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:26:08.625846    6133 generic.go:356] "Generic (PLEG): container finished" podID="10a20102-8426-4a13-a2eb-076c5ae75e1e" containerID="270102a10128925c01cfea7a72c10abcc5b59f8ddb3b46716ca123d41d25d735" exitCode=0
Oct 30 22:26:08.626330 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:26:08.625889    6133 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-67fvs/must-gather-cr4s2" event={"ID":"10a20102-8426-4a13-a2eb-076c5ae75e1e","Type":"ContainerDied","Data":"270102a10128925c01cfea7a72c10abcc5b59f8ddb3b46716ca123d41d25d735"}
Oct 30 22:26:08.626330 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:26:08.626216    6133 scope.go:122] "RemoveContainer" containerID="270102a10128925c01cfea7a72c10abcc5b59f8ddb3b46716ca123d41d25d735"
Oct 30 22:26:09.526385 ip-10-0-143-246 kubenswrapper[6133]: E1030 22:26:09.526352    6133 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6jppj" containerName="extractor"
Oct 30 22:26:09.647299 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:26:09.647273    6133 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-67fvs_must-gather-cr4s2_10a20102-8426-4a13-a2eb-076c5ae75e1e/gather/0.log"
Oct 30 22:26:15.276942 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:26:15.276911    6133 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-q7fwz_af83b188-09cb-4621-a9da-be33856d19f0/global-pull-secret-syncer/0.log"
Oct 30 22:26:15.370152 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:26:15.370118    6133 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-must-gather-67fvs/must-gather-cr4s2"]
Oct 30 22:26:15.370338 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:26:15.370305    6133 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-must-gather-67fvs/must-gather-cr4s2" podUID="10a20102-8426-4a13-a2eb-076c5ae75e1e" containerName="copy" containerID="cri-o://ad35f24f3bb95445ff0b48dc1447526640714e17e31930c1712bee2f72c3ba8b" gracePeriod=2
Oct 30 22:26:15.376375 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:26:15.376350    6133 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-must-gather-67fvs/must-gather-cr4s2"]
Oct 30 22:26:15.385331 ip-10-0-143-246 kubenswrapper[6133]: E1030 22:26:15.385295    6133 status_manager.go:1045] "Failed to get status for pod" err="pods \"must-gather-cr4s2\" is forbidden: User \"system:node:ip-10-0-143-246.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-must-gather-67fvs\": no relationship found between node 'ip-10-0-143-246.ec2.internal' and this object" podUID="10a20102-8426-4a13-a2eb-076c5ae75e1e" pod="openshift-must-gather-67fvs/must-gather-cr4s2"
Oct 30 22:26:15.615030 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:26:15.615005    6133 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-67fvs_must-gather-cr4s2_10a20102-8426-4a13-a2eb-076c5ae75e1e/copy/0.log"
Oct 30 22:26:15.617598 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:26:15.617579    6133 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-67fvs/must-gather-cr4s2"
Oct 30 22:26:15.631814 ip-10-0-143-246 kubenswrapper[6133]: E1030 22:26:15.631777    6133 status_manager.go:1045] "Failed to get status for pod" err="pods \"must-gather-cr4s2\" is forbidden: User \"system:node:ip-10-0-143-246.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-must-gather-67fvs\": no relationship found between node 'ip-10-0-143-246.ec2.internal' and this object" podUID="10a20102-8426-4a13-a2eb-076c5ae75e1e" pod="openshift-must-gather-67fvs/must-gather-cr4s2"
Oct 30 22:26:15.655235 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:26:15.655211    6133 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-67fvs_must-gather-cr4s2_10a20102-8426-4a13-a2eb-076c5ae75e1e/copy/0.log"
Oct 30 22:26:15.655622 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:26:15.655594    6133 generic.go:356] "Generic (PLEG): container finished" podID="10a20102-8426-4a13-a2eb-076c5ae75e1e" containerID="ad35f24f3bb95445ff0b48dc1447526640714e17e31930c1712bee2f72c3ba8b" exitCode=143
Oct 30 22:26:15.655696 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:26:15.655643    6133 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-67fvs/must-gather-cr4s2"
Oct 30 22:26:15.655696 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:26:15.655669    6133 scope.go:122] "RemoveContainer" containerID="ad35f24f3bb95445ff0b48dc1447526640714e17e31930c1712bee2f72c3ba8b"
Oct 30 22:26:15.662885 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:26:15.662862    6133 scope.go:122] "RemoveContainer" containerID="270102a10128925c01cfea7a72c10abcc5b59f8ddb3b46716ca123d41d25d735"
Oct 30 22:26:15.669638 ip-10-0-143-246 kubenswrapper[6133]: E1030 22:26:15.669603    6133 status_manager.go:1045] "Failed to get status for pod" err="pods \"must-gather-cr4s2\" is forbidden: User \"system:node:ip-10-0-143-246.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-must-gather-67fvs\": no relationship found between node 'ip-10-0-143-246.ec2.internal' and this object" podUID="10a20102-8426-4a13-a2eb-076c5ae75e1e" pod="openshift-must-gather-67fvs/must-gather-cr4s2"
Oct 30 22:26:15.675310 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:26:15.675294    6133 scope.go:122] "RemoveContainer" containerID="ad35f24f3bb95445ff0b48dc1447526640714e17e31930c1712bee2f72c3ba8b"
Oct 30 22:26:15.675624 ip-10-0-143-246 kubenswrapper[6133]: E1030 22:26:15.675604    6133 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ad35f24f3bb95445ff0b48dc1447526640714e17e31930c1712bee2f72c3ba8b\": container with ID starting with ad35f24f3bb95445ff0b48dc1447526640714e17e31930c1712bee2f72c3ba8b not found: ID does not exist" containerID="ad35f24f3bb95445ff0b48dc1447526640714e17e31930c1712bee2f72c3ba8b"
Oct 30 22:26:15.675699 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:26:15.675635    6133 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ad35f24f3bb95445ff0b48dc1447526640714e17e31930c1712bee2f72c3ba8b"} err="failed to get container status \"ad35f24f3bb95445ff0b48dc1447526640714e17e31930c1712bee2f72c3ba8b\": rpc error: code = NotFound desc = could not find container \"ad35f24f3bb95445ff0b48dc1447526640714e17e31930c1712bee2f72c3ba8b\": container with ID starting with ad35f24f3bb95445ff0b48dc1447526640714e17e31930c1712bee2f72c3ba8b not found: ID does not exist"
Oct 30 22:26:15.675699 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:26:15.675653    6133 scope.go:122] "RemoveContainer" containerID="270102a10128925c01cfea7a72c10abcc5b59f8ddb3b46716ca123d41d25d735"
Oct 30 22:26:15.675897 ip-10-0-143-246 kubenswrapper[6133]: E1030 22:26:15.675882    6133 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"270102a10128925c01cfea7a72c10abcc5b59f8ddb3b46716ca123d41d25d735\": container with ID starting with 270102a10128925c01cfea7a72c10abcc5b59f8ddb3b46716ca123d41d25d735 not found: ID does not exist" containerID="270102a10128925c01cfea7a72c10abcc5b59f8ddb3b46716ca123d41d25d735"
Oct 30 22:26:15.675938 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:26:15.675901    6133 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"270102a10128925c01cfea7a72c10abcc5b59f8ddb3b46716ca123d41d25d735"} err="failed to get container status \"270102a10128925c01cfea7a72c10abcc5b59f8ddb3b46716ca123d41d25d735\": rpc error: code = NotFound desc = could not find container \"270102a10128925c01cfea7a72c10abcc5b59f8ddb3b46716ca123d41d25d735\": container with ID starting with 270102a10128925c01cfea7a72c10abcc5b59f8ddb3b46716ca123d41d25d735 not found: ID does not exist"
Oct 30 22:26:15.740253 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:26:15.740213    6133 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/10a20102-8426-4a13-a2eb-076c5ae75e1e-must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/10a20102-8426-4a13-a2eb-076c5ae75e1e-must-gather-output\") pod \"10a20102-8426-4a13-a2eb-076c5ae75e1e\" (UID: \"10a20102-8426-4a13-a2eb-076c5ae75e1e\") "
Oct 30 22:26:15.740423 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:26:15.740291    6133 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/10a20102-8426-4a13-a2eb-076c5ae75e1e-kube-api-access-bpwt2\" (UniqueName: \"kubernetes.io/projected/10a20102-8426-4a13-a2eb-076c5ae75e1e-kube-api-access-bpwt2\") pod \"10a20102-8426-4a13-a2eb-076c5ae75e1e\" (UID: \"10a20102-8426-4a13-a2eb-076c5ae75e1e\") "
Oct 30 22:26:15.741716 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:26:15.741685    6133 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/10a20102-8426-4a13-a2eb-076c5ae75e1e-must-gather-output" pod "10a20102-8426-4a13-a2eb-076c5ae75e1e" (UID: "10a20102-8426-4a13-a2eb-076c5ae75e1e"). InnerVolumeSpecName "must-gather-output". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:26:15.742568 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:26:15.742540    6133 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/10a20102-8426-4a13-a2eb-076c5ae75e1e-kube-api-access-bpwt2" pod "10a20102-8426-4a13-a2eb-076c5ae75e1e" (UID: "10a20102-8426-4a13-a2eb-076c5ae75e1e"). InnerVolumeSpecName "kube-api-access-bpwt2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:26:15.841762 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:26:15.841678    6133 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bpwt2\" (UniqueName: \"kubernetes.io/projected/10a20102-8426-4a13-a2eb-076c5ae75e1e-kube-api-access-bpwt2\") on node \"ip-10-0-143-246.ec2.internal\" DevicePath \"\""
Oct 30 22:26:15.841762 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:26:15.841708    6133 reconciler_common.go:299] "Volume detached for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/10a20102-8426-4a13-a2eb-076c5ae75e1e-must-gather-output\") on node \"ip-10-0-143-246.ec2.internal\" DevicePath \"\""
Oct 30 22:26:15.977759 ip-10-0-143-246 kubenswrapper[6133]: E1030 22:26:15.977721    6133 status_manager.go:1045] "Failed to get status for pod" err="pods \"must-gather-cr4s2\" is forbidden: User \"system:node:ip-10-0-143-246.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-must-gather-67fvs\": no relationship found between node 'ip-10-0-143-246.ec2.internal' and this object" podUID="10a20102-8426-4a13-a2eb-076c5ae75e1e" pod="openshift-must-gather-67fvs/must-gather-cr4s2"
Oct 30 22:26:16.334984 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:26:16.334955    6133 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-thksb_2bcee03a-9b40-4155-8d81-43dece439162/konnectivity-agent/0.log"
Oct 30 22:26:16.712276 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:26:16.712198    6133 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-143-246.ec2.internal_b29468cdb315509fa1bb87ee196f0663/haproxy/0.log"
Oct 30 22:26:17.355145 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:26:17.355104    6133 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="10a20102-8426-4a13-a2eb-076c5ae75e1e" path="/var/lib/kubelet/pods/10a20102-8426-4a13-a2eb-076c5ae75e1e/volumes"
Oct 30 22:26:22.308008 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:26:22.307962    6133 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-5dc8d7d89b-8727r_17da628d-31f5-456c-ad37-52c884d22cf4/kube-state-metrics/0.log"
Oct 30 22:26:22.429455 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:26:22.429429    6133 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-5dc8d7d89b-8727r_17da628d-31f5-456c-ad37-52c884d22cf4/kube-rbac-proxy-main/0.log"
Oct 30 22:26:22.555598 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:26:22.555573    6133 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-5dc8d7d89b-8727r_17da628d-31f5-456c-ad37-52c884d22cf4/kube-rbac-proxy-self/0.log"
Oct 30 22:26:22.819930 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:26:22.819903    6133 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-77f4f76594-htsvq_9d2cced5-9ceb-421f-b831-8370a5ee0ca9/monitoring-plugin/0.log"
Oct 30 22:26:23.716041 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:26:23.716012    6133 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-df5jt_7ddfa67d-20a0-473d-937f-25fd8e0ba80e/node-exporter/0.log"
Oct 30 22:26:23.837407 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:26:23.837376    6133 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-df5jt_7ddfa67d-20a0-473d-937f-25fd8e0ba80e/kube-rbac-proxy/0.log"
Oct 30 22:26:23.959926 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:26:23.959882    6133 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-df5jt_7ddfa67d-20a0-473d-937f-25fd8e0ba80e/init-textfile/0.log"
Oct 30 22:26:24.085202 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:26:24.085174    6133 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-7bd7489957-f26w7_484e37a0-fd1d-4fd0-a200-939b8945984b/kube-rbac-proxy-main/0.log"
Oct 30 22:26:24.189916 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:26:24.189885    6133 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-must-gather-rsvfp/perf-node-gather-daemonset-trwcb"]
Oct 30 22:26:24.190189 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:26:24.190177    6133 cpu_manager.go:414] "containerMap: removing container" podUID="10a20102-8426-4a13-a2eb-076c5ae75e1e" containerName="gather"
Oct 30 22:26:24.190233 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:26:24.190191    6133 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="10a20102-8426-4a13-a2eb-076c5ae75e1e" containerName="gather"
Oct 30 22:26:24.190233 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:26:24.190206    6133 cpu_manager.go:414] "containerMap: removing container" podUID="10a20102-8426-4a13-a2eb-076c5ae75e1e" containerName="copy"
Oct 30 22:26:24.190233 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:26:24.190211    6133 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="10a20102-8426-4a13-a2eb-076c5ae75e1e" containerName="copy"
Oct 30 22:26:24.190322 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:26:24.190273    6133 memory_manager.go:365] "RemoveStaleState removing state" podUID="10a20102-8426-4a13-a2eb-076c5ae75e1e" containerName="copy"
Oct 30 22:26:24.190322 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:26:24.190286    6133 memory_manager.go:365] "RemoveStaleState removing state" podUID="10a20102-8426-4a13-a2eb-076c5ae75e1e" containerName="gather"
Oct 30 22:26:24.193102 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:26:24.193078    6133 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-rsvfp/perf-node-gather-daemonset-trwcb"
Oct 30 22:26:24.204957 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:26:24.204932    6133 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/4d4d5067-b28d-4548-8223-751308c6f0f7-podres\") pod \"perf-node-gather-daemonset-trwcb\" (UID: \"4d4d5067-b28d-4548-8223-751308c6f0f7\") " pod="openshift-must-gather-rsvfp/perf-node-gather-daemonset-trwcb"
Oct 30 22:26:24.205112 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:26:24.204966    6133 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/4d4d5067-b28d-4548-8223-751308c6f0f7-proc\") pod \"perf-node-gather-daemonset-trwcb\" (UID: \"4d4d5067-b28d-4548-8223-751308c6f0f7\") " pod="openshift-must-gather-rsvfp/perf-node-gather-daemonset-trwcb"
Oct 30 22:26:24.205112 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:26:24.204982    6133 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/4d4d5067-b28d-4548-8223-751308c6f0f7-lib-modules\") pod \"perf-node-gather-daemonset-trwcb\" (UID: \"4d4d5067-b28d-4548-8223-751308c6f0f7\") " pod="openshift-must-gather-rsvfp/perf-node-gather-daemonset-trwcb"
Oct 30 22:26:24.205248 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:26:24.205153    6133 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t96qw\" (UniqueName: \"kubernetes.io/projected/4d4d5067-b28d-4548-8223-751308c6f0f7-kube-api-access-t96qw\") pod \"perf-node-gather-daemonset-trwcb\" (UID: \"4d4d5067-b28d-4548-8223-751308c6f0f7\") " pod="openshift-must-gather-rsvfp/perf-node-gather-daemonset-trwcb"
Oct 30 22:26:24.205306 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:26:24.205243    6133 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4d4d5067-b28d-4548-8223-751308c6f0f7-sys\") pod \"perf-node-gather-daemonset-trwcb\" (UID: \"4d4d5067-b28d-4548-8223-751308c6f0f7\") " pod="openshift-must-gather-rsvfp/perf-node-gather-daemonset-trwcb"
Oct 30 22:26:24.207332 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:26:24.207314    6133 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-7bd7489957-f26w7_484e37a0-fd1d-4fd0-a200-939b8945984b/kube-rbac-proxy-self/0.log"
Oct 30 22:26:24.220876 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:26:24.220821    6133 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-rsvfp\"/\"default-dockercfg-8wgdx\""
Oct 30 22:26:24.221053 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:26:24.221034    6133 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-rsvfp\"/\"kube-root-ca.crt\""
Oct 30 22:26:24.221109 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:26:24.221090    6133 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-rsvfp\"/\"openshift-service-ca.crt\""
Oct 30 22:26:24.221638 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:26:24.221622    6133 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-rsvfp/perf-node-gather-daemonset-trwcb"]
Oct 30 22:26:24.306001 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:26:24.305963    6133 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4d4d5067-b28d-4548-8223-751308c6f0f7-sys\") pod \"perf-node-gather-daemonset-trwcb\" (UID: \"4d4d5067-b28d-4548-8223-751308c6f0f7\") " pod="openshift-must-gather-rsvfp/perf-node-gather-daemonset-trwcb"
Oct 30 22:26:24.306177 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:26:24.306091    6133 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4d4d5067-b28d-4548-8223-751308c6f0f7-sys\") pod \"perf-node-gather-daemonset-trwcb\" (UID: \"4d4d5067-b28d-4548-8223-751308c6f0f7\") " pod="openshift-must-gather-rsvfp/perf-node-gather-daemonset-trwcb"
Oct 30 22:26:24.306177 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:26:24.306106    6133 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/4d4d5067-b28d-4548-8223-751308c6f0f7-podres\") pod \"perf-node-gather-daemonset-trwcb\" (UID: \"4d4d5067-b28d-4548-8223-751308c6f0f7\") " pod="openshift-must-gather-rsvfp/perf-node-gather-daemonset-trwcb"
Oct 30 22:26:24.306177 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:26:24.306146    6133 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/4d4d5067-b28d-4548-8223-751308c6f0f7-proc\") pod \"perf-node-gather-daemonset-trwcb\" (UID: \"4d4d5067-b28d-4548-8223-751308c6f0f7\") " pod="openshift-must-gather-rsvfp/perf-node-gather-daemonset-trwcb"
Oct 30 22:26:24.306177 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:26:24.306158    6133 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/4d4d5067-b28d-4548-8223-751308c6f0f7-podres\") pod \"perf-node-gather-daemonset-trwcb\" (UID: \"4d4d5067-b28d-4548-8223-751308c6f0f7\") " pod="openshift-must-gather-rsvfp/perf-node-gather-daemonset-trwcb"
Oct 30 22:26:24.306177 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:26:24.306166    6133 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/4d4d5067-b28d-4548-8223-751308c6f0f7-lib-modules\") pod \"perf-node-gather-daemonset-trwcb\" (UID: \"4d4d5067-b28d-4548-8223-751308c6f0f7\") " pod="openshift-must-gather-rsvfp/perf-node-gather-daemonset-trwcb"
Oct 30 22:26:24.306343 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:26:24.306213    6133 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/4d4d5067-b28d-4548-8223-751308c6f0f7-proc\") pod \"perf-node-gather-daemonset-trwcb\" (UID: \"4d4d5067-b28d-4548-8223-751308c6f0f7\") " pod="openshift-must-gather-rsvfp/perf-node-gather-daemonset-trwcb"
Oct 30 22:26:24.306343 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:26:24.306215    6133 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/4d4d5067-b28d-4548-8223-751308c6f0f7-lib-modules\") pod \"perf-node-gather-daemonset-trwcb\" (UID: \"4d4d5067-b28d-4548-8223-751308c6f0f7\") " pod="openshift-must-gather-rsvfp/perf-node-gather-daemonset-trwcb"
Oct 30 22:26:24.306343 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:26:24.306215    6133 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t96qw\" (UniqueName: \"kubernetes.io/projected/4d4d5067-b28d-4548-8223-751308c6f0f7-kube-api-access-t96qw\") pod \"perf-node-gather-daemonset-trwcb\" (UID: \"4d4d5067-b28d-4548-8223-751308c6f0f7\") " pod="openshift-must-gather-rsvfp/perf-node-gather-daemonset-trwcb"
Oct 30 22:26:24.327639 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:26:24.327612    6133 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t96qw\" (UniqueName: \"kubernetes.io/projected/4d4d5067-b28d-4548-8223-751308c6f0f7-kube-api-access-t96qw\") pod \"perf-node-gather-daemonset-trwcb\" (UID: \"4d4d5067-b28d-4548-8223-751308c6f0f7\") " pod="openshift-must-gather-rsvfp/perf-node-gather-daemonset-trwcb"
Oct 30 22:26:24.331158 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:26:24.331134    6133 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-7bd7489957-f26w7_484e37a0-fd1d-4fd0-a200-939b8945984b/openshift-state-metrics/0.log"
Oct 30 22:26:24.469486 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:26:24.469412    6133 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_f77c55ab-9c27-4472-ab3b-83c11b803db0/prometheus/0.log"
Oct 30 22:26:24.504800 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:26:24.504772    6133 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-rsvfp/perf-node-gather-daemonset-trwcb"
Oct 30 22:26:24.591421 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:26:24.591393    6133 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_f77c55ab-9c27-4472-ab3b-83c11b803db0/config-reloader/0.log"
Oct 30 22:26:24.716430 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:26:24.716403    6133 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_f77c55ab-9c27-4472-ab3b-83c11b803db0/thanos-sidecar/0.log"
Oct 30 22:26:24.794651 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:26:24.794625    6133 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-rsvfp/perf-node-gather-daemonset-trwcb"]
Oct 30 22:26:24.795941 ip-10-0-143-246 kubenswrapper[6133]: W1030 22:26:24.795918    6133 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod4d4d5067_b28d_4548_8223_751308c6f0f7.slice/crio-83d6ca97267f7e6a629511ae79455fe66d3770b60d0bbb22a2638a32c31ebd63 WatchSource:0}: Error finding container 83d6ca97267f7e6a629511ae79455fe66d3770b60d0bbb22a2638a32c31ebd63: Status 404 returned error can't find the container with id 83d6ca97267f7e6a629511ae79455fe66d3770b60d0bbb22a2638a32c31ebd63
Oct 30 22:26:24.847240 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:26:24.847206    6133 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_f77c55ab-9c27-4472-ab3b-83c11b803db0/kube-rbac-proxy-web/0.log"
Oct 30 22:26:24.997304 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:26:24.997276    6133 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_f77c55ab-9c27-4472-ab3b-83c11b803db0/kube-rbac-proxy/0.log"
Oct 30 22:26:25.122973 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:26:25.122898    6133 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_f77c55ab-9c27-4472-ab3b-83c11b803db0/kube-rbac-proxy-thanos/0.log"
Oct 30 22:26:25.252226 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:26:25.252201    6133 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_f77c55ab-9c27-4472-ab3b-83c11b803db0/init-config-reloader/0.log"
Oct 30 22:26:25.692462 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:26:25.692427    6133 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-rsvfp/perf-node-gather-daemonset-trwcb" event={"ID":"4d4d5067-b28d-4548-8223-751308c6f0f7","Type":"ContainerStarted","Data":"ae9f92972f14584aa4ebcaf6b6e1b3c9f9185cdfa5f369ead7749d55f109ef44"}
Oct 30 22:26:25.692663 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:26:25.692479    6133 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-rsvfp/perf-node-gather-daemonset-trwcb" event={"ID":"4d4d5067-b28d-4548-8223-751308c6f0f7","Type":"ContainerStarted","Data":"83d6ca97267f7e6a629511ae79455fe66d3770b60d0bbb22a2638a32c31ebd63"}
Oct 30 22:26:25.692663 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:26:25.692575    6133 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-rsvfp/perf-node-gather-daemonset-trwcb"
Oct 30 22:26:25.738797 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:26:25.738746    6133 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-must-gather-rsvfp/perf-node-gather-daemonset-trwcb" podStartSLOduration=1.738730637 podStartE2EDuration="1.738730637s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 22:26:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:26:25.738231091 +0000 UTC m=+1892.498986980" watchObservedRunningTime="2026-10-30 22:26:25.738730637 +0000 UTC m=+1892.499486525"
Oct 30 22:26:26.167756 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:26:26.167711    6133 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-868d6f78df-nx2k6_fbd8f09b-9962-4910-8b25-267cbbb3ea5d/thanos-query/0.log"
Oct 30 22:26:26.296510 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:26:26.296484    6133 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-868d6f78df-nx2k6_fbd8f09b-9962-4910-8b25-267cbbb3ea5d/kube-rbac-proxy-web/0.log"
Oct 30 22:26:26.427641 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:26:26.427565    6133 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-868d6f78df-nx2k6_fbd8f09b-9962-4910-8b25-267cbbb3ea5d/kube-rbac-proxy/0.log"
Oct 30 22:26:26.553149 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:26:26.553122    6133 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-868d6f78df-nx2k6_fbd8f09b-9962-4910-8b25-267cbbb3ea5d/prom-label-proxy/0.log"
Oct 30 22:26:26.677928 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:26:26.677857    6133 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-868d6f78df-nx2k6_fbd8f09b-9962-4910-8b25-267cbbb3ea5d/kube-rbac-proxy-rules/0.log"
Oct 30 22:26:26.801386 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:26:26.801360    6133 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-868d6f78df-nx2k6_fbd8f09b-9962-4910-8b25-267cbbb3ea5d/kube-rbac-proxy-metrics/0.log"
Oct 30 22:26:31.705161 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:26:31.705135    6133 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-rsvfp/perf-node-gather-daemonset-trwcb"
Oct 30 22:26:34.205551 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:26:34.205527    6133 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-fqmbc_209b4e58-e6c2-48f1-8589-f7b6be79720b/dns/0.log"
Oct 30 22:26:34.332759 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:26:34.332726    6133 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-fqmbc_209b4e58-e6c2-48f1-8589-f7b6be79720b/kube-rbac-proxy/0.log"
Oct 30 22:26:34.732098 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:26:34.732075    6133 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-wwb9f_fc4a67d9-8c7c-4f71-9a8a-2bba09329bc6/dns-node-resolver/0.log"
Oct 30 22:26:36.329099 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:26:36.329070    6133 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-qzgkj_fe472529-1294-46a1-a666-dc00d5d37eca/node-ca/0.log"
Oct 30 22:26:39.402655 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:26:39.402622    6133 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-fl5mx_4621048c-230e-441f-85fc-423e4ba525ec/serve-healthcheck-canary/0.log"
Oct 30 22:26:39.527685 ip-10-0-143-246 kubenswrapper[6133]: E1030 22:26:39.527649    6133 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6jppj" containerName="extractor"
Oct 30 22:26:40.479973 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:26:40.479946    6133 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-6jppj_6192d2be-bba7-4099-b54c-e7e9b8d75389/kube-rbac-proxy/0.log"
Oct 30 22:26:40.602959 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:26:40.602933    6133 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-6jppj_6192d2be-bba7-4099-b54c-e7e9b8d75389/exporter/0.log"
Oct 30 22:26:40.729085 ip-10-0-143-246 kubenswrapper[6133]: I1030 22:26:40.729049    6133 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-6jppj_6192d2be-bba7-4099-b54c-e7e9b8d75389/extractor/0.log"